I currently work in the capacity of Software Engineer at a reputed company in Sri Lanka. I'm most keen in J2EE Technologies and love working with open source libraries which fit my project needs. Very interested in the NoSQL concept and experimenting with various products to find a good blend for our projects within the company. Part time freelancer. Avid contributor in the stackoverflow arena. Android is another area which I am looking at with very keen interest as I believe mobile development is the way forward. Dinuka is a DZone MVB and is not an employee of DZone and has posted 24 posts at DZone. You can read more from them at their website. View Full User Profile

JBoss clustering Quick Reference Guide

04.18.2011
| 9194 views |
  • submit to reddit

So i was involved in clustering two JBoss instances and for some reason i had lost my guide i made for myself last time i did a clustering implementation. This time i thought ill blog it so as to keep it as a reference for myself as well as anyone who might need to quickly do JBoss Clustering as we all know how time critical all IT projects are. I will put this in a step by step approach to make it easier to read and comprehend. Note that this approach assumes you are clustering the all configuration in JBoss.

  • Locate the cluster-service.xml located within the deploy folder. Open it and locate the attribute PartitionName within the first mbean you see in that file and change the value ${jboss.partition.name:DefaultPartition} to ${jboss.partition.name}. As in delete the part DefaultParition. 
  •  Find the mbean code which equals to org.jboss.ha.jndi.HANamingService and org.jboss.ha.hasessionstate.server.HASessionStateService. Within that locate the tag   . Chane the value jboss:service=${jboss.partition.name:DefaultPartition} to jboss:service=${jboss.partition.name}. Also add the following to both the mbeans mentioned above ${jboss.partition.name}.
  • In your application jndi reference make sure to change the port to 1100 because in a clustered environment HA-JNDI(High availability JNDI) is used to look up resource. So localhost:1099 should be localhost:1100. Also you have to give the jndi in a comma separated way indicating all servers being clustered. This is needed in a case where if one server goes down still look ups will not fail as you have specified all servers. Say for example you have two servers named ABC and XYZ. Then if you only specify your jndi url as ABC:1100, this will work but in the event that ABC goes down all your other look ups will also fail. So to avoid that kind of situations make sure to define it as ABC:1100,XYZ:1100. This will make sure to try the other server in the case one fails.
  • In all your EJBs make sure to put the Annotation @Clustered(partition="mypartition"). The name we specify should be specified in your run script which i will give in the end of this article.
  • If you running on Redhat make sure the HA-JNDI port is open by issuing the following commands to open port 1100;

iptables -A INPUT -i eth0 -p tcp --sport 1100 -m state --state ESTABLISHED -j ACCEPT

/etc/init.d/iptables restart


service iptables restart

Next i would like to get your attention on a draw back in JNDI failover and how to overcome it. Say for example you have two servers named serverA and serverB, If serverA shuts down then serverB should handle the whole load. This works out of the box from JBoss. But if serverA starts again and serverB goes down then you are screwed because you will be getting an error as java.lang.RuntimeException: Unreachable?: Service unavailable.

To overcome this there is a work around. Open the file standardjboss.xml locate in the all/conf directory. Within that file locate the text clustered-stateless-rmi-invoker. Now within that you will see number of elements. In one of these you will find something called org.jboss.proxy.ejb.SingleRetryInterceptor. You will find two instances of this. You need to change it to org.jboss.proxy.ejb.RetryInterceptor. What this does is it will keep retrying to connect infinitely until the other server comes up. Ofcourse this is not a perfect solution but just a work around. Note that this is with respect to Jboss 4.2.3 that i am talking about. This might have been fixed in later releases which im not aware of.

So the change will look as below;

<invoker-proxy-binding>
<name>clustered-stateless-rmi-invoker</name>
<invoker-mbean>jboss:service=invoker,type=jrmpha</invoker-mbean>
<proxy-factory>org.jboss.proxy.ejb.ProxyFactoryHA</proxy-factory>
<proxy-factory-config>
<client-interceptors>
<home>
<interceptor>org.jboss.proxy.ejb.HomeInterceptor</interceptor>
<interceptor>org.jboss.proxy.SecurityInterceptor</interceptor>
<interceptor>org.jboss.proxy.TransactionInterceptor</interceptor>
<interceptor>org.jboss.proxy.ejb.RetryInterceptor</interceptor>
<interceptor call-by-value="false">org.jboss.invocation.InvokerInterceptor</interceptor>
<interceptor call-by-value="true">org.jboss.invocation.MarshallingInvokerInterceptor</interceptor>
</home>
<bean>
<interceptor>org.jboss.proxy.ejb.StatelessSessionInterceptor</interceptor>
<interceptor>org.jboss.proxy.SecurityInterceptor</interceptor>
<interceptor>org.jboss.proxy.TransactionInterceptor</interceptor>
<interceptor>org.jboss.proxy.ejb.RetryInterceptor</interceptor>
<interceptor call-by-value="false">org.jboss.invocation.InvokerInterceptor</interceptor>
<interceptor call-by-value="true">org.jboss.invocation.MarshallingInvokerInterceptor</interceptor>
</bean>
</client-interceptors>
</proxy-factory-config>
</invoker-proxy-binding>

Lastly i present to you the jboss_redhat_init.sh file which sets the jboss partition name to be used across the application which is the same name used in the @Clustered annotation. 
#!/bin/sh
#
# $Id: jboss_init_redhat.sh 71252 2008-03-25 17:52:00Z dbhole $
#
# JBoss Control Script
#
# To use this script run it as root - it will switch to the specified user
#
# Here is a little (and extremely primitive) startup/shutdown script
# for RedHat systems. It assumes that JBoss lives in /usr/local/jboss,
# it's run by user 'jboss' and JDK binaries are in /usr/local/jdk/bin.
# All this can be changed in the script itself.
#
# Either modify this script for your requirements or just ensure that
# the following variables are set correctly before calling the script.

#define where jboss is - this is the directory containing directories log, bin, conf etc
JBOSS_HOME=${JBOSS_HOME:-"/opt/xyz/jboss-4.2.3.GA"}

#define the user under which jboss will run, or use 'RUNASIS' to run as the current user
JBOSS_USER=${JBOSS_USER:-"RUNASIS"}

#make sure java is in your path
JAVAPTH=${JAVAPTH:-"/usr/java/jdk1.6.0_16/bin"}

#configuration to use, usually one of 'minimal', 'default', 'all'
JBOSS_CONF=${JBOSS_CONF:-"all"}

#if JBOSS_HOST specified, use -b to bind jboss services to that address
JBOSS_BIND_ADDR=${JBOSS_HOST:+"-b $JBOSS_HOST"}

#define the script to use to start jboss
JBOSSSH=${JBOSSSH:-"$JBOSS_HOME/bin/run.sh -b 172.2.23.2 -c $JBOSS_CONF $JBOSS_BIND_ADDR -Djboss.partition.name=mypartition"}

if [ "$JBOSS_USER" = "RUNASIS" ]; then
SUBIT=""
else
SUBIT="su - $JBOSS_USER -c "
fi

if [ -n "$JBOSS_CONSOLE" -a ! -d "$JBOSS_CONSOLE" ]; then
# ensure the file exists
touch $JBOSS_CONSOLE
if [ ! -z "$SUBIT" ]; then
chown $JBOSS_USER $JBOSS_CONSOLE
fi
fi

if [ -n "$JBOSS_CONSOLE" -a ! -f "$JBOSS_CONSOLE" ]; then
echo "WARNING: location for saving console log invalid: $JBOSS_CONSOLE"
echo "WARNING: ignoring it and using /dev/null"
JBOSS_CONSOLE="/dev/null"
fi

#define what will be done with the console log
JBOSS_CONSOLE=${JBOSS_CONSOLE:-"/dev/null"}

JBOSS_CMD_START="cd $JBOSS_HOME/bin; $JBOSSSH"

if [ -z "`echo $PATH | grep $JAVAPTH`" ]; then
export PATH=$PATH:$JAVAPTH
fi

if [ ! -d "$JBOSS_HOME" ]; then
echo JBOSS_HOME does not exist as a valid directory : $JBOSS_HOME
exit 1
fi

echo JBOSS_CMD_START = $JBOSS_CMD_START

function procrunning() {
procid=0
JBOSSSCRIPT=$(echo $JBOSSSH | awk '{print $1}' | sed 's/\//\\\//g')
for procid in `/sbin/pidof -x "$JBOSSSCRIPT"`; do
ps -fp $procid | grep "${JBOSSSH% *}" > /dev/null && pid=$procid
done
}


stop() {
pid=0
procrunning
if [ $pid = '0' ]; then
echo -n -e "\nNo JBossas is currently running\n"
exit 1
fi

RETVAL=1

# If process is still running

# First, try to kill it nicely
for id in `ps --ppid $pid | awk '{print $1}' | grep -v "^PID$"`; do
if [ -z "$SUBIT" ]; then
kill -15 $id
else
$SUBIT "kill -15 $id"
fi
done

sleep=0
while [ $sleep -lt 120 -a $RETVAL -eq 1 ]; do
echo -n -e "\nwaiting for processes to stop";
sleep 10
sleep=`expr $sleep + 10`
pid=0
procrunning
if [ $pid == '0' ]; then
RETVAL=0
fi
done

# Still not dead... kill it

count=0
pid=0
procrunning

if [ $RETVAL != 0 ] ; then
echo -e "\nTimeout: Shutdown command was sent, but process is still running with PID $pid"
exit 1
fi

echo
exit 0
}

case "$1" in
start)
cd $JBOSS_HOME/bin
if [ -z "$SUBIT" ]; then
eval $JBOSS_CMD_START >${JBOSS_CONSOLE} 2>&1 &
else
$SUBIT "$JBOSS_CMD_START >${JBOSS_CONSOLE} 2>&1 &"
fi
;;
stop)
stop
;;
restart)
$0 stop
$0 start
;;
*)
echo "usage: $0 (start|stop|restart|help)"
esac

The part to note here is -Djboss.partition.name=mypartition which sets the name of the cluster as an application wide property.

Thats it guys. If you have any issues or improvements please leave a comment which is as always much appreciated. 

From http://dinukaroshan.blogspot.com/2011/03/jboss-clustering-quick-reference-guide.html

Published at DZone with permission of Dinuka Arseculeratne, author and DZone MVB.

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)

Tags:

Comments

Abhishek Chavan replied on Thu, 2011/04/21 - 2:58am

Great Article. I loved the bit about the Retry logic. Didn't know that piece

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.