Circuit Breaker Pattern
In this tutorial, we are going to discuss Design Patterns of Microservices Architecture which is the Circuit Breaker Pattern. We will use this pattern and practice when designing microservice architecture.
Have you ever wondered, why there are circuit breakers in your house? If your house is powered by electricity, there must be circuit breakers since they control and protect the wiring circuits that feed outlets in your home. When the electricity comes through the main grid, it always flows through these circuit breakers. Therefore, if the main grid behaves in an abnormal way, or if additional power tries to flow, then the particular circuit breaker will switch off. As a result, the internal wiring circuits of your house will be protected. So, this behavior of circuit breakers is very similar to how the Circuit Breaker Design Pattern works.
The Circuit Breaker pattern is a popular design pattern used in Microservices Architecture, that falls under the Sustainable Design Patterns category. In Microservices architecture, a service usually calls other services to retrieve data, and there is the chance that the downstream service may be down. It may be because of slow network connection, timeouts, or temporal unavailability. Therefore, retrying calls can solve the issue. However, if there is a severe issue with a particular microservice, then it will be unavailable for a longer time. In such a case, the request will be continuously sent to that service, since the client doesn’t have any knowledge about a particular service being down. As a result, the network resources will be exhausted with low performance and a bad user experience. Also, the failure of one service might lead to Cascading failures throughout the application.
Therefore, you can use the Circuit Breaker Pattern to overcome this problem. With the help of this pattern, the client will invoke a remote service through a proxy. This proxy will basically behave as an electrical circuit breaker. So, when the number of failures crosses the threshold number, the circuit breaker trips for a particular time period. Then, all the attempts to invoke the remote service will fail within this timeout period. After the timeout expires, the circuit breaker allows a limited number of test requests to pass through it. If those requests succeed, the circuit breaker resumes back to the normal operation. Otherwise, if there is a failure, the timeout period begins again.
The Circuit Breaker pattern has three states:
In this state, the Circuit Breaker routes the requests to the Microservice and counts the number of failures in each period of time. That means it works without any failures. But if the number of failures in a certain period of time exceeds a threshold, the circuit will trip and will move to an “Open” state.
When the Circuit breaker moves to the “Open” state, requests from the Microservices will fail immediately, and an exception will be returned. However, after a timeout, the Circuit Breaker will go to the “Half-Open” state.
In this state, the Circuit Breaker allows only a limited number of requests from the Microservice, to pass through and invoke the operation. If these requests are successful, the Circuit Breaker will go back to the “Closed” state. However, if any request fails again, it goes back to the “Open” state.
Use Case of Circuit Breaker Pattern
Let’s take an example to understand, where we can apply Circuit Breaker Pattern in Microservices architecture.
Assume there are 5 different services in a Microservices application. Whenever it receives requests, the server will allocate one thread to call the particular service. But, due to some failure, the service is a little delayed, and the thread is waiting. However, it’s okay, if only one thread is waiting for that service. But, if the service is a high demanding service that gets many requests, it is not good to hold. Because more threads will be allocated for this service within some time, these threads will have to wait.
As a result, the remaining requests that come to your service will be blocked or queued. Even though the service is recovered back, the webserver is still trying to process the requests that are in the queue. Because the webserver will never recover since it receives requests continuously. Eventually, this might lead to Cascading failures throughout the application. Therefore, this kind of scenario will lead to a crash on your services and even the application.
The above scenario is a perfect example to apply Circuit Breaker Pattern. Assume you have a defined threshold for a particular service, as it should respond within 200ms. As I mentioned above, that service is a high-demand service, that continuously receives requests. In case, if 75% of those requests are reaching the upper threshold (150ms — 200ms) means that the service is going to fail soon. However, if several requests exceed the maximum threshold (200ms) means the service not responding anymore. As a result, it will fail back to the consumer and inform this particular service is not available. So, if you remember the above-mentioned states of this pattern, now we are moving to the “Open” state from the “Closed” state.
As a result, all those requests that come to the particular service, not going to wait anymore. However, after a timeout, the Circuit Breaker sends ping requests to that service in the background. That means now we are in the “Half-Open” state of the Circuit Breaker pattern. If these requests are successful, the Circuit Breaker will allow sending requests for that service again.
So, you can use the Circuit Breaker Pattern to improve the fault-tolerance and resilience of the Microservice Architecture and also to prevent the cascading of failure to other microservices.