Servlet 3.1/Spring MVC Non Blocking IO

Servlet 3.0 was released as part of Java EE 6 and made huge changes focused

at ease-of-use.The idea was to leverage the latest language features such as

annotations and generics and modernize how Servlets can be written.One of the

major change was Async Servlets. The web.xml was also made as optional as

possible. Servlet 3.1 released as part of Java EE 7, was an incremental release

focusing on couple of key features including Non-blocking IO.

Non-blocking I/O - Servlet 3.0 (Async Servlets as discussed in previous article)

allowed asynchronous request processing but only traditional I/O was permitted

which is blocking. This can restrict scalability of your applications.

Non-blocking I/O allow to build scalable applications.

Let’s discuss what I mean by above. We have learnt in previous article that in

case of async servlet, we must use non blocking code.So Let’s modify our

earlier MyServlet Code and replace runnable logic as below:-

Let’s revisit the code snippet which we discussed in previous article-

@WebServlet(name="myServlet", urlPatterns={"/asyncprocess"}, asyncSupported=true)

public class MyServlet extends HttpServlet {

   public void doGet(HttpServletRequest request, HttpServletResponse response) {

OutputStream out = response.getOutputStream();

          AsyncContext aCtx = request.startAsync(request, response);

          doAsyncREST(request).thenAccept(json -> {

out.write(json);  ---> Blocking!

ctx.complete();

          });

}

In above code, Container Request Thread is released and work is done in another

thread.So for async Servlet to actually work as expected, we have below requirement:

1. doAsyncREST() must use Async library to call REST and return CompletableFuture. This  is possible using AsyncHttpClient which we have already used in previous article.

2. thenAccept() should use async libraries.

But in Servlet 3.0, IO was traditional blocking and hence the thread calling out.write()

will block.

Let’s say, we have to write a huge json back to client.As we are using NIO

connector. OutputStream will first write to buffers and those buffers need to

be emptied by clients(using selector/channel mechanism of NIO). Now, if clients

are on slow network, then out.write() will have to wait till buffers are empty again

as InputStream/OutputStream is blocking.

Above problem of blocking was removed by Servlet 3.1 release by introducing async IO.

Servlet 3.1 Async IO

Let’s discuss this with the help of a code snippet as below:

void doGet(request, response) {

       ServletOutputStream out = response.getOutputStream();

       AsyncContext ctx = request.startAsync();

       out.setWriteListener(new WriteListener() {

           void onWritePossible() {

               while (out.isReady()) {

                   byte[] buffer = readFromSomeSource();

                   if (buffer != null)

                       out.write(buffer); ---> Async Write!

                   else{

                       ctx.complete(); break;

                   }

                 }

               }

           });

       }

In above code, we are making use of Write/Read Listener which were introduced

in 3.1. WriteListener is an interface which has onWritePossible() method which gets

called by Servlet Container. ServletOutputStreamt.isReady() is used to check if it is

possible to write in NIO channel buffers. In case it returns false then it schedules a

call on Servlet Container for onWritePossible() method and at some later point,

onWritePossible() is called on another thread.So, in this way, out.write() never block

for slow client to empty channel buffers.

Non Blocking IO in Spring?

To use this feature of non blocking IO in Spring application, we would need Spring 5

which is has JAVA EE 7 as baseline version. So , our earlier example which is also

mentioned below will execute in completely non blocking mode if we run this code

on Spring 5 MVC,Tomcat 8.5+

@GetMapping(value = "/asyncNonBlockingRequestProcessing")

   public CompletableFuture asyncNonBlockingRequestProcessing(){

           ListenableFuture listenableFuture = getRequest.execute(new AsyncCompletionHandler() {

               @Override

               public String onCompleted(Response response) throws Exception {

                   logger.debug("Async Non Blocking Request processing completed");

                   return "Async Non blocking...";

               }

           });

           return listenableFuture.toCompletableFuture();

   }

By now, we have discussed on how Servlet has evolved along with Spring to

provide complete non-blocking support which mean we can scale our application

with very less number of threads. In Next articles, we would be discussing on

Spring reactive stack(i.e. Spring webflux).One might think that, if Spring MVC is

capable of handling request in Non-blocking way, then why Spring webflux is

released as a separate stack and was it actually needed?

Stay tuned, it would be a interesting read in coming articles.