Introduction to Load Balancing
What is a Load Balancer?
When dealing with the scalability of our software systems, we may often come across the challenge of dealing with an influx of requests. For example, imagine running a popular web app that sells dog apparel. We just released a new product (a fashionable dog sweater) and have noticed a significant increase of requests hitting our single server. Users really want to purchase our new dog sweater for their pups! Right now, our web app architecture looks something like this:
In this architecture, our single server is becoming a bottleneck and is putting our application at risk of performing suboptimally. In order to alleviate the load on our single server, we decide to scale our web app horizontally and purchase a few additional servers. Each server will host a replica of our app now we will be able to distribute request load more effectively. However, with multiple servers, we have the following issue:
We need a way to direct the traffic! Our app won’t know where to send the requests unless we provide guidance on which server to send the request to. This is where a load balancer comes into play.
A load balancer is a piece of hardware or software (and sometimes both) that helps distribute requests between different system resources. Load balancers are not just an essential aspect when scaling a system horizontally; they also help prevent specific system resources from getting overloaded and possibly going offline. In addition, load balancers are flexible enough to be placed in various places in a software systems architecture. In our web app example, since we are primarily trying to distribute the load between our servers, here is what our new web app architecture will look like with a load balancer:
When we examine the above image, the way requests route to individual servers for our web app may seem a bit like magic. How exactly is the load balancer deciding which server is best fit to handle the incoming request? How does the load balancer make sure one server doesn’t end up taking all the requests by accident? These questions are all decided based on the load balancing algorithm that the load balancer uses. Let’s explore what these algorithms are and how they work!
Before we move on, try this question: