Demystifying Load Balancing in Go: A Comprehensive Guide

Load balancing is a crucial technique used to distribute incoming traffic across multiple backend servers, ensuring optimal performance, scalability, and reliability. In this article, we will explore load balancing in Go and implement three popular load balancing algorithms: Round Robin, Least Connections, and Random. We will also discuss the differences between layer 4 and layer 7 load balancing.

Table of Contents:

Load Balancing Fundamentals
Layer 4 vs. Layer 7 Load Balancing
Round Robin Load Balancer
Least Connections Load Balancer
Random Load Balancer
Implementation in Go
Conclusion

1. Load Balancing Fundamentals

Load balancing involves distributing incoming requests across multiple backend servers to achieve better performance, scalability, and fault tolerance. It ensures that no single server is overwhelmed with traffic while maintaining high availability and efficient resource utilization.

There are two primary types of load balancing:

Layer 4 Load Balancing:

Layer 4 load balancing operates at the transport layer (TCP/UDP) of the OSI model. It focuses on distributing traffic based on IP addresses, ports, and transport protocols. Layer 4 load balancers make routing decisions based on network-level information without inspecting application-layer protocols.

Layer 7 Load Balancing:

Layer 7 load balancing, also known as application-level load balancing, operates at the application layer of the OSI model. It performs deep packet inspection, allowing load balancers to make routing decisions based on the content, URL, cookies, or other application-specific data. Layer 7 load balancers are more intelligent and can distribute traffic based on application-specific needs.

3. Round Robin Load Balancer:

The Round Robin load balancing algorithm distributes requests in a sequential, circular manner. Each request is assigned to the next available backend server in the rotation. It ensures an even distribution of requests across all servers. However, Round Robin does not consider server loads or capacities, potentially leading to uneven resource utilization.

4. Least Connections Load Balancer

The Least Connections load balancing algorithm distributes requests to the backend server with the fewest active connections. It aims to evenly distribute the workload by considering the current connections on each server. This algorithm ensures that traffic is directed to servers with lighter loads, enabling better utilization of resources.

5. Random Load Balancer

The Random load balancing algorithm randomly selects a backend server for each incoming request. It is the simplest load-balancing algorithm but does not consider server loads or capacities. Random load balancing can be effective when backend servers have similar capabilities, but it may lead to uneven distribution if servers have varying capacities or loads.

6. Implementation in Go

LoadBalancer the interface specifies the ServeHTTP method and the GetNextAvailableServer method. The Server struct represents a backend server, including its URL, health status, weight, and current number of connections.

  
// LoadBalancer defines the interface for a load balancer.
type LoadBalancer interface {
	ServeHttp(w http.ResponseWriter, r *http.Request)
	GetNextAvailableServer() *Server
}

// Server represents a backend server.
type Server struct {
	URL         string
	Alive       bool
	Weight      int
	Connections int
	mutex       sync.Mutex // using it to protect concurrent access to alive and connections field
}

The ReverseProxy struct represents a reverse proxy for a specific backend server. The NewReverseProxy function creates a new instance of the ReverseProxy struct with the specified backend URL. The ServeHTTP method of the ReverseProxy struct forwards the incoming request to the backend server.

  
type ReverseProxy struct {
	backendURL string
	proxy      *httputil.ReverseProxy
}

func NewReverseProxy(backendURL string) *ReverseProxy {
	backend, _ := url.Parse(backendURL)

	return &ReverseProxy{
		backendURL: backendURL,
		proxy:      httputil.NewSingleHostReverseProxy(backend),
	}

}

// Forwards the incoming request to backend server
func (rp *ReverseProxy) ServerHttp(w http.ResponseWriter, r *http.Request) {
	fmt.Printf("Forwarding request to %s : %s\n", rp.backendURL, r.URL.Path)
	rp.proxy.ServeHTTP(w, r)
}

The RoundRobinLoadBalancer struct maintains a list of servers and keeps track of the next server to use for load balancing.

The ServeHTTP method of RoundRobinLoadBalancer distributes incoming requests in a round-robin manner. It calls the GetNextAvailableServer method to obtain the next available server and creates a reverse proxy for that server’s URL. The reverse proxy then forwards the request to the backend server.

  
package main

import (
	"net/http"
)

type RoundRobinLB struct {
	servers []*Server
	next    int
}

func NewRoundRobinLB(servers []*Server) *RoundRobinLB {
	return &RoundRobinLB{
		servers: servers,
		next:    0,
	}
}

// GetNextAvailableServer returns the next available backend server in a round-robin manner.
func (lb *RoundRobinLB) GetNextAvailableServer() *Server {

	numServers := len(lb.servers)

	// Start searching from the next index
	start := lb.next

	for i := 0; i < numServers; i++ {
		serverIndex := (start + i) % numServers
		server := lb.servers[serverIndex]

		server.mutex.Lock()
		alive := server.Alive
		server.mutex.Unlock()

		if alive {
			// update the next index for next iteration
			lb.next = (serverIndex + 1) % numServers
			return server
		}
	}
	// No available servers found, return nil
	return nil
}
func (lb *RoundRobinLB) ServeHTTP(w http.ResponseWriter, r *http.Request) {
	server := lb.GetNextAvailableServer()
	if server != nil {
		proxy := NewReverseProxy(server.URL)

		logger.Print("server is ", server)
		// Set the logger for the ReverseProxy
		proxy.ServerHttp(w, r)
	} else {
		logger.Print("server is ", server)
		// TODO:
		// Handle the case when no available server is found
	}
}

The LeastConnectionsLoadBalancer struct represents a load balancer that distributes incoming requests to the backend server with the fewest active connections. The NewLeastConnectionsLoadBalancer function creates a new instance of the LeastConnectionsLoadBalancer struct with the given list of servers.

The ServeHTTP method of LeastConnectionsLoadBalancer distributes incoming requests to the backend server with the fewest active connections. It calls the GetNextAvailableServer method to obtain the server with the fewest connections and creates a reverse proxy for that server’s URL. The reverse proxy then forwards the request to the backend server.

  
package main

import (
	"net/http"
)

type LeastConnectionLB struct {
	servers []*Server
}

func NewLeastConnectionLB(servers []*Server) *LeastConnectionLB {
	return &LeastConnectionLB{
		servers: servers,
	}
}

// ServeHTTP distributes the incoming request to the backend server with the fewest active connections.
func (lb *LeastConnectionLB) ServeHTTP(w http.ResponseWriter, r *http.Request) {
	server := lb.GetNextAvailableServer()
	logger.Print("server is %v", server)
	proxy := NewReverseProxy(server.URL)
	proxy.ServerHttp(w, r)
}

// GetNextAvailableServer returns the backend server with the fewest active connections.
func (lb *LeastConnectionLB) GetNextAvailableServer() *Server {
	minConn := -1
	var selectedServer *Server
	for _, server := range lb.servers {
		server.mutex.Lock()
		alive := server.Alive
		connections := server.Connections
		server.mutex.Unlock()

		if !alive {
			continue
		}
		if minConn == -1 || connections < minConn {
			minConn = connections
			selectedServer = server
		}
	}

	if selectedServer != nil {
		selectedServer.mutex.Lock()
		selectedServer.Connections++
		selectedServer.mutex.Unlock()
		return selectedServer
	}

	// No available servers found, return nil
	return nil
}

The RandomLoadBalancer struct represents a load balancer that distributes incoming requests to a random backend server. The NewRandomLoadBalancer function creates a new instance of the RandomLoadBalancer struct with the given list of servers.

The ServeHTTP method of RandomLoadBalancer distributes incoming requests to a random backend server. It calls the GetNextAvailableServer method to obtain a random available server and creates a reverse proxy for that server’s URL. The reverse proxy then forwards the request to the backend server.

  
package main

import (
	"math/rand"
	"net/http"
)

type RandomLB struct {
	servers []*Server
}

func NewRandomLB(servers []*Server) *RandomLB {
	return &RandomLB{
		servers: servers,
	}
}

// ServeHTTP distributes the incoming request to a random backend server.
func (lb *RandomLB) ServeHTTP(w http.ResponseWriter, r *http.Request) {
	server := lb.GetNextAvailableServer()
	if server != nil {
		proxy := NewReverseProxy(server.URL)
		logger.Print("server is ", server)
		proxy.ServerHttp(w, r)

	} else {
		logger.Print("server is ", server)
		//TODO:
		// Handle the case when no available server is found
	}
}

// GetNextAvailableServer returns a random backend server.
func (lb *RandomLB) GetNextAvailableServer() *Server {
	var availableServers []*Server

	for _, server := range lb.servers {
		server.mutex.Lock()

		if server.Alive {
			availableServers = append(availableServers, server)
		}
		server.mutex.Unlock()
	}
	if len(availableServers) == 0 {
		// No available servers found, return nil
		return nil
	}
	// return some random server from available servers

	return availableServers[rand.Intn(len(availableServers))]
}

Now, let’s set up our main function to start the load balancer. In the main.go file, we define the backend servers as a slice of Server instances. Then, we create instances of the load balancers: RoundRobinLoadBalancer , LeastConnectionsLoadBalancer , and RandomLoadBalancer . We register each load balancer as an HTTP handler for a specific route. Finally, we start the HTTP server on port 8080.

  
package main

import (
	"fmt"
	"log"
	"net/http"
	"os"
)

var logger *log.Logger

func init() {
	// Create the logger with desired settings
	logger = log.New(os.Stdout, "", log.LstdFlags)
}

func main() {
	// Define the backend servers
	servers := []*Server{{
		URL: "https://jsonplaceholder.typicode.com", Weight: 1, Alive: true},
		{URL: "https://httpbin.org", Weight: 2, Alive: true},
		{URL: "https://reqres.in", Weight: 3, Alive: true},
	}

	// Create the load balancers
	roundRobinLB := NewRoundRobinLB(servers)
	leastConnectionLB := NewLeastConnectionLB(servers)
	randomLB := NewRandomLB(servers)

	// Register the load balancers as HTTP handlers
	http.Handle("/round-robin", roundRobinLB)
	http.Handle("/least-connections", leastConnectionLB)
	http.Handle("/random", randomLB)

	// Start the server
	fmt.Println("Load balancers started.")

	err := http.ListenAndServe(":8080", nil)
	if err != nil {
		fmt.Printf("Error starting server: %s\n", err.Error())
	}
}

How to run the server is mentioned in the README file. You can clone the full code from my GitHub repository.

7. Conclusion

Load balancing plays a crucial role in modern distributed systems, ensuring optimal performance, scalability, and reliability. In this article, we explored load-balancing fundamentals and implemented three popular load-balancing algorithms in Go: Round Robin, Least Connections, and Random. We also discussed the differences between layer 4 and layer 7 load balancing.

By understanding load-balancing algorithms and their implementations in Go, you can build efficient and scalable applications that can handle growing traffic demands. Whether you choose Round Robin, Least Connections, or Random load balancing, it’s important to consider your specific requirements and the characteristics of your backend servers.

Post converted from Medium by ZMediumToMarkdown.

Demystifying Load Balancing in Go: A Comprehensive Guide