This article explains a number multi threaded patterns and concepts.

Thread safety

Thread pool

Server Programs such as web servers repeatedly execute requests from multiple clients and these are oriented around processing a large number of short tasks.Creating a new request every time a request is received would be very expensive in some programming laguages like java. A thread pool is a collection of pre created threads. Since the thread already exists in threadpool when the request arrives, the delay introduced by thread creation is eliminated, making the application more responsive. Advantages of thread pool are

Difference between concurrency and parallelism

Concurrency is when two or more tasks can start, run, and complete in overlapping time periods. It doesn't necessarily mean they'll ever both be running at the same instant. For example, multitasking on a single-core machine.Concurrency as a property of a program or system

Parallelism is when tasks literally run at the same time, e.g., on a multicore processor. Parallelism (in the sense of multithreading) is not possible with single core processors.

What are synchronous, asynchronous , blocking and non blocking calls

Synchronous - Thread will complete the action, either by success or failure. ie the work is done in the thread that calls the function and the method does not return until it is finished. 

Blocking - Thread will wait on action untill success or failure . Failure is commonly a timeout.Examples would be 

Non-blocking - Thread will not wait to complete the action. Non blocking io is a very good example of non blocking calls. Event loop in nodejs is non blocking in nature..(non blocking sockets using epoll). Note that there is no call back function like asynchronous calls hence polling is required.

Asynchronous - Another thread/threadpool/process will complete the action . The action may simply be added to queue which is being processed .In a nutshell there is delegation. ie asynchronous methods return immediately because another thread does the work and raises a flag or fires an event  or executes a call back function when the work is done. Asynchnous calls are very common , for example

TPC , TPR ,Event loop (not fully reactive) ,Fully reactive non blocking event loop

Note on reactive programming: Reactive programming promotes event driven,non blocking approach to data processing . Reactive programming involves modeling data and events as observable data streams and implementing data processing routines to react to the changes in those streamsReactive programming style can be used for event loops. Note that reactive programming only promotes non blocking behaviour, it does not guratee it. As allready mentioned reactive is a programming style(there can be a library in any programming language) . Note that one confusing thing is that when articles use the term reactive component  it is often meant non blocking component, not necessarily coded in reactive style. Hence reactive and non blocking are often used synonymously eg reactive database.

Event loop in java : Java frameworks like netty, vertx, jboss undertow , Spring Webflux are based on signle threaded event loop. Spring Webflux by default has Netty as an embedded server. Netty and Undertow are non-servlet runtimes and Tomcat and Jetty are well-known servlet containers. Before the introduction of Spring Webflux, Spring MVC used to have Tomcat as its default embedded server, which uses the Thread Per Request Model.  Now, with Webflux, Netty has been preferred as a default, which uses the Event Loop Model.  Although Webflux is also supported on Tomcat, it's only with versions that implement Servlet 3.1 or later APIs as serverlet 3.1 introduced NIO . Servlet 3.0 provided Async Servlets which allowed asynchronous request processing, but only the (traditional /blocking) I/O was permitted. This can restrict that scalability of your applications . Bascially what happens is that servlet thread can dissociate itself from request , delegate task to a throttled thread pool, and then return to servlet thread pool for handling http requests to webserver. Servelt 3.0 Asynch servelet was useful for handling a situation where long running tasks could exhaust the servlet thread pool. 

 Servlet 3.1 permitted non blocking io .Read more AND


http multiplexing

Question> What is http multiplexing and it decrease server load?

Multiplexing helps to reduce server load by reducing number of connections but in TPR model  (more connections != more server threads)



Double checked locking pattern

Double checked design pattern is a concurrency design pattern used to reduce locking over head while implementing lazy initialization / singleton pattern . The code should first check if the lock has to be acquired before acquring the lock.

Here is an implementation of singleton pattern without double checking pattern.

class Singleton {
    private Singleton singleton;
   /* this introduces an locking over head every time getInstance() is called  */
    public synchronized Singleton getInstance() {
        if (singleton == null) {
            singleton = new Singleton();
        return singleton;

As acquring a lock can introduce a significant over head , this implementation is suboptimal.

Here is the implementation of singleton pattern with double checking pattern.

// Note that volatile key word does not work under Java 1.4
class Singleton {
    private volatile static Singleton instance;
    public static Singleton getInstance()
        if (instance == null) {
            /*The lock will be acquired only once when instance is not created, on all subsequent calls to getInstance()
               lock is not acquired.
            synchronized (Singleton.class)
                if (instance == null) {
                    instance = new Singleton();
        return instance;

Note the use of volatile keyword. Volatile Keyword is applicable to variables. volatile keyword in Java guarantees that value of the volatile variable will always be read from main memory and not from Thread's local cache. 


Fan-out/fan-in refers to the pattern of executing multiple functions concurrently and then performing some aggregation on the results. The fanout pattern could be used for better utlization of multiple cores of cpu or in case of io intensive application performing blocking calls.


Read more