Kui Zhang is a Sun Microsystems certified enterprise architect working in a large International IT consulting company Kui has posted 1 posts at DZone. View Full User Profile

Design Flexible and Scalable Online Task Processing Systems in a Java EE 6 Web Container

04.29.2010
| 8145 views |
  • submit to reddit

It was painful for web application architects and designers to handle long running online tasks in web containers until servlet asynchronous processing (Servlet 3.0 specification) was introduced. This article will introduce a solution to leverage this technical edge to create a flexible task processing engine without losing the scalability of the web container.

Long running tasks such as a long web service call or a long running query used to block the current thread on the pending request for a long time. Therefore it could significantly downgrade tje web container’s throughput and reduce the scalability of the system. There are some other solutions as discussed in the attached resources, but that solution needs the EJB container’s help; therefore, in the case of a web container like Tomcat, it is not applicable.

My solution is based on Servlet specification 3.0 standards, part of Java EE 6 release. The following diagram illustrates this architecture.

 

                        

      

Architecture Diagram

 

The Task Engine mainly consists of a Thread Pool.  In order to improve task processing capacity, a thread pool is created to handle multiple tasks concurrently, and the thread pool also allows having a Blocking Queue to hold tasks waiting for processing.  This Task Engine actually delegates its task processing to this thread pool, and provides concurrent task submissions initiated from web clients.

 

 

 

 Task Engine Class Diagram

Design Considerations

1.         Usage of ServletContext listener. This is the right place to hook up this multithreaded task engine to the web container, since it is loosely coupled with the servlet to carry out recycling on attributes and allocated thread pool objects; and make it available in the application scope.  Besides, with the annotation @WebListener, we could make it pluggable to any servlet.

 

2.         Separation of task processing logic from others. The Task interface is an abstract layer above concrete processing in every thread in the thread pool. The methods beforeRun() and afterRun() in this interface are designed to put servlet related logic like request/response handling over there while pure task processing is designated to the inherited method Run(), therefore extensibility and reusability is achieved to some extend.

 

3.         Communication to Client. With Ajax and asynchronous processing support, users won’t be blocked from further interaction on web pages after submits a long running task, and this request handling in the servlet container won’t be blocked until the requested resource become available any more. Servlet container is able to recycle this request thread to serve other requests and later when necessary this request could be resumed to push back data got from task processing process to the client.

         When the asyncSupported attribute is set to true for a servlet, the response object is not committed on method exit. Calling startAsync() returns an AsyncContext object that caches the request/response object pair. The AsyncContext object is then served as a task context to initialize a task object and put into the application-scoped task engine waiting for processing. The original request thread is then recycled. When task processing ends or task is rejected by task engine, we could either call AsyncContext.getResponse().getWriter().print(...), and then complete() to commit the response, or calling dispatch()to direct the flow to another JSP page as a result.

 

4.         Policy for dealing with overload considerations. In most cases the usage of unbounded queue in thread Pool is enough for web server.  New tasks will queue up if there is no idle thread in the thread pool, which could lead to smooth out transient burst of tasks efficiently.  But for some cases for example, where there is a significant request peak time in your system everyday or the back end task processing is experiencing of slowing down for quite a while, this unbounded queue may keep growing, which may exhaust resources and kill system finally. Then the bounded queue is coming to rescue. The task engine with a bounded queue could be designed to reject any new task after it is full, which makes system work quite stably. In addition, the most important thing to make this strategy work is that with the help of AsynContext and Ajax the client could be acknowledged immediately about this rejection by telling that system is experiencing high volume now; please resubmit your task later. This is acceptable in most cases from client perspective since they are allowed to continue working on other pages meanwhile.

         There are four policy implementations provided for thread pool totally in Java 6 when a bounded work queue fills up: AbortPolicy, CallerRunsPolicy, DiscardPolicy, and DiscardOldestPolicy. The default policy abort, causes execute to throw the unchecked Rejected-ExecutionException. Our system is taking this policy, and then catches this exception and implement our own overflow handling here. Please refer to the following state diagram for details.  The CallerRunsPolicy allows the thread that invokes execute itself runs the task. This provides a simple feedback control mechanism that will slow down the rate that new tasks are submitted. The discard policy silently discards the newly submitted task if it cannot be queued for execution; the discard-oldest policy discards the task that would otherwise be executed next and tries to resubmit the new task.


State Diagram for Task Processing Based on Abort Policy

 

 

Sample Code Snippets (Adopting Abort Policy):

 @WebListener( )
public class TaskEngineListener implements ServletContextListener {


public void contextInitialized(ServletContextEvent sce) {

logger.info("Start Task Engine...");

TaskEngine taskengine= new TaskEngine();

sce.getServletContext().setAttribute("TaskEngine", taskengine);

}



public void contextDestroyed(ServletContextEvent sce) {

logger.info("Try to shut down Task Engine");

ServletContext ctx= sce.getServletContext();

TaskEngine taskengine=(TaskEngine)ctx.getAttribute("TaskEngine");

if(taskengine!=null){

taskengine.shutdown();

}
ctx.removeAttribute("TaskEngine");
}

}

 @WebServlet (asyncSupported = true)

public class AsyncServlet extends HttpServlet {



@Override

protected void doPost(HttpServletRequest request, HttpServletResponse response)

throws ServletException, IOException {



response.setContentType("text/html;charset=UTF-8");

response.setHeader("Cache-Control", "private");

response.setHeader("Pragma", "no-cache");

final AsyncContext ac = request.startAsync(request, response);

ac.setTimeout(10 * 60 * 1000);



Task task= new AsyncServletTask(ac);

TaskEngine taskengine = (TaskEngine) request.getServletContext().getAttribute("TaskEngine");

if (!(taskengine.addTask(task)))

{ PrintWriter out = null;

try {

String name = ac.getRequest().getParameter("name");

out = ac.getResponse().getWriter();

out.println("Task: " + name + " is declined, Please submit later");

out.close();

} catch (IOException ex) {

logger.log(Level.SEVERE, "Task Rejection Exception",ex);

} finally {

ac.complete();

}

}

}



}


public class TaskEngine {

private final static int POOLMAX=4;

private final static int TASKQUEUELENGTH=3;

private final ExecutorService jobExecutor

= new TaskEngineThreadPool(POOLMAX, POOLMAX,

0L, TimeUnit.MILLISECONDS,

new LinkedBlockingQueue<Runnable>(TASKQUEUELENGTH), new ThreadPoolExecutor.AbortPolicy());



public TaskEngine(){

}



public synchronized boolean addTask(Task task){

boolean done=true;

try{

jobExecutor.execute(task);



}catch(RejectedExecutionException ex){

done=false;



}

return done;

}

...

}

 

   public class AsyncServletTask implements Task {

private static final Logger logger = Logger.getLogger(AsyncServletTask.class.getName());

private AsyncContext ac=null;

private String taskName=null;

public AsyncServletTask(AsyncContext in){

ac=in;



}

@Override

public void run() {

//handle task processing here



}



@Override

public void beforeRun() {

this.taskName=ac.getRequest().getParameter("name");



}



@Override

public void afterRun() {

try {

PrintWriter out = ac.getResponse().getWriter();

//logger.info("Get Task:"+taskName);

out.println("Servlet AsyncServlet "+taskName+" response");

out.close();

} catch(IOException ex) {

logger.log(Level.SEVERE,"AfterRun() exception",ex);

}

ac.complete();



}


}

In Conclusion

This article introduces a solution design to handle online task processing under web container with consideration of system scalability and extensibility. The advantages of this solution are:

  1. Make servlet container manage its threads more efficiently when handling long running tasks.
  2. Lightweight task engine implementation here allows a centralized and customized control over long running tasks, which improves system efficiency overall
  3. Provide flexible processing acknowledgement to the client and therefore make user page more responsive.

Disadvantages:

  1. Thread Pool tuning is suggested.

 

About the author

Kui Zhang is a Sun Microsystems Certified Enterprise Architect working in an International IT consulting company. He has architected, designed and developed many J2EE/JEE applications on Websphere and Weblogic

 

 

Resources:

  1. http://java.sun.com/javase/6/docs/api/java/util/concurrent/ThreadPoolExecutor.html
  2. http://blogs.sun.com/enterprisetechtips/entry/asynchronous_support_in_servlet_3
  3. http://www.javaranch.com/journal/2004/03/AsynchronousProcessingFromServlets.html
  4. http://www.javaworld.com/javaworld/jw-02-2009/jw-02-servlet3.html
  5. Java Concurrency in Practice   Author: Brian Goetz

 

Published at DZone with permission of its author, Kui Zhang.

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)

Comments

Andy Leung replied on Fri, 2010/04/30 - 9:01am

I remember I've read some Head First J2EE Book or from Sun recommending not to use ThreadPool in J2EE Container, would you provide more details what is the impact if you do that?

Thanks in advance.

Janet Schiller replied on Fri, 2010/04/30 - 2:58pm

I am very interested in this article, good design! The web container delegates its task processing works to this thread pool actually.

Jeff Szeto replied on Sun, 2010/05/02 - 4:26pm in response to: Andy Leung

I have the same question in my mind too while reading the article. The article doesn't quite address how to leverage thread management from the container in this particular design (kinda misguide a bit when I read the article title "... in a JEE 6 web container").

However, it is much easier to deal with thread management this day with jdk concurrency package, so it may not be a big deal anymore. Thoughts?

Kui Zhang replied on Mon, 2010/05/03 - 10:16am

Mulththreading with web container is challenging, since each user request is served by container managed thread. In the user request and session scope, thread creation from yourself is not recommended since it is not easy to limit your thread size totally and manage their lifecycles in order not to exaust web container's resources. However, in the application scope, it is easier to do so if you need to . If you look at Servlet spec 3 with its samples from Sun Microsystems, you cold get better understanding about this. Hope this help.

Denis Zwoo replied on Thu, 2010/05/13 - 7:04pm

Was following your example but got stuck with async processing of doPost on GFv3. 

Have no problems with doGet but doPost displays a strange behavior:

Almost every consecutive call results in a 30sec time lag before the server sends a response. Calling doGet after a suspended doPost results in the same delay as well. It looks like the container doesn't release the resource or maybe I am missing something...

Do you have any ideas what can cause this?

Here's a very simple servlet to illustrate this.

package app;

import java.io.*;
import javax.servlet.*;
import javax.servlet.annotation.*;
import javax.servlet.http.*;

@WebServlet(urlPatterns = "/app", asyncSupported = true)
public class AppOne extends HttpServlet {

@Override
protected void doGet(HttpServletRequest req, HttpServletResponse res) throws ServletException, IOException {
final AsyncContext actx = req.startAsync(req, res);
actx.start(new Runnable() {

public void run() {
try {
HttpServletResponse res = (HttpServletResponse)actx.getResponse();
res.setContentType("text/plain");
PrintWriter out = res.getWriter();
out.println("doGet-async");
out.flush();
} catch (IOException ex) {
System.err.println(ex.getMessage());
} finally {
System.out.println("doGet ASYNC FINISHED");
actx.complete();
}
}
});
}

@Override
protected void doPost(HttpServletRequest req, HttpServletResponse res) throws ServletException, IOException {
System.out.println("doPost");
final AsyncContext actx = req.startAsync(req, res);
actx.start(new Runnable() {

public void run() {
try {
HttpServletResponse res = (HttpServletResponse)actx.getResponse();
res.setContentType("text/plain");
res.setHeader("Cache-Control", "private");
res.setHeader("Pragma", "no-cache");

PrintWriter out = res.getWriter();
out.println("doPost-async");
out.flush();
} catch (IOException ex) {
System.err.println(ex.getMessage());
} finally {
System.out.println("doPost ASYNC FINISHED");
actx.complete();
}
}
});

}
}

The servlet gets called from index.html with a form:

<form method="post" action="/exSimpleAsync/app">
<input type="text" size="20" name="email" value="hello">
<input type="submit" name="login" value="Click me">
</form>

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.