Java/Springboot Blocking vs Non-Blocking REST API Implementation
In this article, we will walk through building a simple REST API that performs the User lookup in both the traditional Blocking approach and the Non-blocking approach using Java and Springboot. On the way, we will learn about Java 8 key features like Future and CompletableFuture that aids us to build Non-blocking APIs.
Traditional REST APIs deployed on Tomcat or any container are built in a blocking fashion, which means that for each HTTP request, a thread is assigned and the thread is held for the whole lifecycle of the request as shown below.
With the Blocking approach, containers like Tomcat would not be able to support a huge number of concurrent requests impacting scalability due to the fact that Tomcat has a fixed-size thread pool. With more concurrent requests, requests have to wait in the queue to get a thread to execute. You would not notice any issue with this approach if your scalability requirements are simple for example supporting 500–1000 concurrent users. When load increases, you will start seeing delays impacting performance numbers.
With a Non-Blocking approach, you can design your application in such a way that the subsequent calls to different classes/methods, calls to external systems to run in a separate thread thus freeing the current thread to do other tasks or return to the container pool. Once the thread finishes its execution, it can notify the initial code that called it via callbacks which will be executed by a different thread and finally HTTP response is returned. As you see, still a lot of thread execution but nothing is blocked and threads are free to handle many tasks parallelly thus reducing the need for a larger Container thread pool or the current thread pool can handle many concurrent requests.
Java 8 had some awesome features like streams, lambda and much more. Among them, there has been considerable improvement on libraries like threading, asynchronous processing, a more mature way of handling callbacks, failures, errors and chaining asynchronous calls. CompletableFuture class is our primary candidate that handles all of these concerns.
I am not going to go deep on CompletableFuture here as we have tons of articles/blogs out there. I will show a simple example of how this can be leveraged in a typical SpringBoot application.
As shown in the figures above, typical flow in any Spring application is Controller -> Service -> Repository/Dao -> Database.
At a high level below are the changes required to leverage this Non-Blocking approach
- The container will have a default thread pool to handle tasks. Define a separate thread pool that will be used by Async calls and any task submitted/tracked by CompletableFuture as below. In this example, we are defining a pool with 1000 threads.
@Bean
public Executor asyncExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(1000);
executor.setMaxPoolSize(1000);
executor.setQueueCapacity(1000);
executor.setThreadNamePrefix("Async-");
executor.initialize();
return executor;
}2. Make the controller return the results wrapped in CompletableFuture
public CompletableFuture<List<UserDto>> getUserByName(@PathVariable String firstName) {
return userService.getUserByNameAsync(firstName);
}3. Make the Service to run in a different thread using @Async annotation and return results in CompletableFuture.
@Async
public CompletableFuture<List<UserDto>> getUserByNameAsync(String firstName){
CompletableFuture<List<User>> users = userRepo.findAllByLastName(firstName);
return users.thenApply( urs -> {
return urs.stream().map(this::userToUserDto).collect(Collectors.toList());
});
}4. Update the Repository call to run in the different thread using @Async annotation.
@Async
public CompletableFuture<List<User>> findAllByLastName(String name);The attached sample application creates users and exposes REST API to search Users by name. Apache has a cool tool Apache Benchmark which I have used to demonstrate the load and the response time around different percentiles for both the approaches.
Created 10000 users using below
$ ab -n 10000 -c 100 -p ./create_user.txt -T application/json http://localhost:8080/springboot-async/user/create
Blocking API — Response times for consuming the API to get users.4000 total requests with 150 concurrent requests.
Non-Blocking API — Response times for consuming the API to get users. 4000 total requests with 150 concurrent requests.
As you see above for the 99th percentile, Blocking API had 4553 ms vs Non-Blocking API had 1870 ms which has an improvement of more than 60%.
I hope this will make you curious and research more about the Non-Blocking approach and new Java 8 Threading features.
Complete code is available @ https://github.com/vimalma1093/springboot-async
Happy coding.
