DZone: Your Personal Tech Universe Javalobby
Published on Javalobby (http://java.dzone.com)
GridGain: Master-Worker in Peer-To-Peer Architecture
By dsetrakyan
Created 2008/05/16 - 8:54am

GridGain - Grid Computing Made Simple [1]We sometimes get questions from users on how to ensure Master-Worker pattern within peer-to-peer (P2P) architecture in GridGain [2]. When designing our API and our deployment model, we purposely went with P2P architecture because we wanted to have ultimate freedom on how a grid node is used. As a result, in GridGain a node can act as master or worker or both, depending on your configuration. Moreover, you don't even have to change a single line of code to get this to work.

The following example shows on how it can be done. In GridGain every node has a notion of attributes which it gets at startup. Here is an example that shows how a node can get a "worker" attribute from Spring XML configuration file at startup:

<bean 
id="grid.cfg"
class="org.gridgain.grid.GridConfigurationAdapter"
scope="singleton">
...
<property name="userAttributes">
<map>
<!-- Make this node a worker node. -->
<entry key="segment.worker" value="true"/>
</map>
</property>
...
</bean>
Now, we need to make sure that only worker nodes are passed into GridTask.map(...) method on the master nodes. To do this, on the master nodes we need to configure GridAttributesTopologySpi [3], the purpose of which is to filter nodes based on their attributes. Here is how the configuration will look like:
<bean 
id="grid.cfg"
class="org.gridgain.grid.GridConfigurationAdapter"
singleton="true">
...
<property name="topologySpi">
<bean class="org.gridgain.grid.spi.topology.attributes.GridAttributesTopologySpi">
<property name="attributes">
<map>
<!-- Include only worker nodes. -->
<entry key="segment.worker" value="true"/>
</map>
</property>
</bean>
</property>
...
</bean>

That's it! To verify that it works, we can add assertion into our GridTask [4] implementation to make sure that all included nodes are indeed "worker" nodes as follows:

public class FooBarGridTask 
extends GridTaskAdapter<String, String> {
...
public Map<GridJob, GridNode> map(
List<GridNode> topology, String arg) {

Map<GridJob, GridNode> jobs =
new HashMap<GridJob, GridNode>(topology.size());

for (GridNode node : topology) {
String workerAttr =
node.getAttribute("segment.worker");

// Assert that worker attribute is present and
// is assigned value "true".
assert workerAttr != null;
assert Boolean.getBoolean(workerAttr) == true;

jobs.put(new FooBarWorkerJob(arg), node);
}

return jobs;
}
...
}

Note, that although we only segmented grid into 2 segments, masters and workers, you can configure as many segments as you like by providing additional node attributes. For example, you can have several worker groups each responsible for processing only a certain subset of jobs.

For addtional examples take a look at Segmenting Grid Nodes [5] article on our Wiki [6].


Source URL: http://java.dzone.com/news/gridgain-master-worker-peer-pe

Links:
[1] http://www.gridgain.com
[2] http://www.gridgain.com
[3] http://www.gridgainsystems.com:8080/wiki/display/GG15UG/GridAttributesTopologySpi
[4] http://www.gridgain.com/javadoc/org/gridgain/grid/GridTask.html
[5] http://www.gridgainsystems.com/wiki/display/GG15UG/Segmenting Grid Nodes
[6] http://wiki.gridgain.org