Enterprise Integration Zone is brought to you in partnership with:

I am an open source and cloud enthusiast. I have over 10 years of experience in leading and implementing several enterprise grade solutions in travel, pharmacy, health care industries. I am currently working as a Software Architect at SciSpike, Inc where we help our clients adopt best enterprise software development/governance practices through consulting, training and software development services. When not working and having fun with my family, I find time to blog key ideas that I discover when solving challenging problems. Ashwini is a DZone MVB and is not an employee of DZone and has posted 6 posts at DZone. You can read more from them at their website. View Full User Profile

ActiveMQ - Network of Brokers Explained

03.12.2014
| 6162 views |
  • submit to reddit

Objective

This 7 part blog series is to share about how to create network of ActiveMQ brokers in order to achieve high availability and scalability.

Why network of brokers?

ActiveMQ message broker is a core component of messaging infrastructure in an enterprise. It needs to be highly available and dynamically scalable to facilitate communication between dynamic heterogeneous distributed applications which have varying capacity needs. 
Scaling enterprise applications on commodity hardware is a rage nowadays. ActiveMQ caters to that very well by being able to create a network of brokers to share the load. 
Many times applications running across geographically distributed data centers need to coordinate messages. Running message producers and consumers across geographic regions/data centers can be architected better using network of brokers.  
ActiveMQ uses transport connectors over which it communicates with message producers and consumers. However, in order to facilitate broker to broker communication, ActiveMQ uses network connectors
A network connector is a bridge between two brokers which allows on-demand message forwarding. 
In other words, if Broker B1 initiates a network connector to Broker B2 then the messages on a channel (queue/topic) on B1 get forwarded to B2 if there is at least one consumer on B2 for the same channel. If the network connector was configured to be duplex, the messages get forwarded from B2 to B1 on demand. 
This is very interesting because it is now possible for brokers to communicate with each other dynamically. 
In this 7 part blog series, we will look into the following topics to gain understanding of this very powerful ActiveMQ feature:
    1. Network Connector Basics - Part 1
    2. Duplex network connectors - Part 2 
    3. Load balancing consumers on local/remote brokers - Part 3
    4. Load-balance consumers/subscribers on remote brokers  
      1. Queue: Load balance remote concurrent consumers - Part 4
      2. Topic: Load Balance Durable Subscriptions on Remote Brokers - Part 5
    5. Store/Forward messages and consumer failover - Part 6
      1. How to prevent stuckmessages
    6. Virtual Destinations - Part 7
To give credit where it is due, the following URLs have helped me in creating this blog post series. 
  1. Advanced Messaging with ActiveMQ by Dejan Bosanac [Slides 32-36] 
  2. Understanding ActiveMQ Broker Networks by Jakub Korab 

Prerequisites

  1. ActiveMQ 5.8.0 – To create broker instances
  2. Apache Ant – To run ActiveMQ sample producer and consumers for demo.
We will use multiple ActiveMQ broker instances on the same machine for the ease of demonstration.

Network Connector Basics - Part 1

The following diagram shows how a network connector functions. It bridges two brokers and is used to forward messages from Broker-1 to Broker-2 on demand if established by Broker-1 to Broker-2. 
A network connector can be duplex so messages could be forwarded in the opposite direction; from Broker-2 to Broker-1, once there is a consumer on Broker-1 for a channel which exists in Broker-2. More on this in Part 2

Setup network connector between broker-1 and broker-2

  • Create two broker instances, say broker-1 and broker-2
Ashwinis-MacBook-Pro:bin akuntamukkala$ pwd
/Users/akuntamukkala/apache-activemq-5.8.0/bin
Ashwinis-MacBook-Pro:bin akuntamukkala$ ./activemq-admin create ../bridge-demo/broker-1

Ashwinis-MacBook-Pro:bin akuntamukkala$ ./activemq-admin create ../bridge-demo/broker-2
Since we will be running both brokers on the same machine, let's configure broker-2 such that there are no port conflicts. 

  • Edit /Users/akuntamukkala/apache-activemq-5.8.0/bridge-demo/broker-2/conf/activemq.xml
    1. Change transport connector to 61626 from 61616
    2. Change AMQP port from 5672 to 6672 (won't be using it for this blog)
  • Edit /Users/akuntamukkala/apache-activemq-5.8.0/bridge-demo/broker-2/conf/jetty.xml
    1. Change web console port to 9161 from 8161
  • Configure Network Connector from broker-1 to broker-2
    Add the following XML snippet to/Users/akuntamukkala/apache-activemq-5.8.0/bridge-demo/broker-1/conf/activemq.xml
Published at DZone with permission of Ashwini Kuntamukkala, author and DZone MVB. (source)

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)