Load balance backends with Nginx upstream

Spread traffic across several app instances or servers with an Nginx upstream block, choose round robin, least_conn or ip_hash, and add passive health checks and keepalive.

  • Advanced
  • Linux VPS
  • 5 min read

By RunRacks Team · Published

When one app process or one server is not enough, Nginx can share requests between several of them. You list the backends in an upstream block, pick a balancing method, and Nginx stops sending traffic to any backend that keeps failing. This guide covers the methods available in open source Nginx, health checks, sticky sessions and connection reuse.

Before you start

  • A RunRacks Linux VPS running Nginx as the front end. See Set up Nginx as a reverse proxy for the basics.
  • Two or more backends serving the same app. They can be several processes on one server (ports 3001, 3002, 3003) or other VPS servers.
  • If backends are on other servers, connect them over a private network or restrict their app port to the load balancer's IP with a firewall.
  • Your app must not keep user state in memory only. Sessions should be in a shared store such as Redis or a database, or you need sticky sessions.

A basic upstream

Create a config file:

Terminal
sudo nano /etc/nginx/conf.d/app-upstream.conf
NGINX
upstream app_pool {
    server 10.0.0.11:8080;
    server 10.0.0.12:8080;
    server 10.0.0.13:8080;
}

Then point your site's location at it:

NGINX
location / {
    proxy_pass http://app_pool;
    proxy_set_header Host              $host;
    proxy_set_header X-Real-IP         $remote_addr;
    proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
}

With no method set, Nginx uses round robin: each request goes to the next server in turn.

Choose a balancing method

Add one of these as the first line inside upstream:

  • Round robin (default): fine when every request costs about the same.
  • `least_conn;` sends each request to the backend with the fewest active connections. Better when some requests are slow, such as reports or uploads.
  • `ip_hash;` always sends the same client IP to the same backend. A simple form of sticky sessions.
  • `hash $cookie_sessionid consistent;` picks a backend by any value you choose, here a session cookie. consistent limits reshuffling when you add or remove a server.

Example:

NGINX
upstream app_pool {
    least_conn;
    server 10.0.0.11:8080;
    server 10.0.0.12:8080;
}

Weights and backup servers

Give stronger servers a bigger share with weight:

NGINX
upstream app_pool {
    server 10.0.0.11:8080 weight=3;
    server 10.0.0.12:8080 weight=1;
    server 10.0.0.13:8080 backup;
}

Here the first server gets roughly three requests for every one the second gets. The backup server only receives traffic when all the others are unavailable. backup cannot be used together with ip_hash or hash.

To take a server out for maintenance without deleting the line, add down and reload.

Passive health checks

Open source Nginx does not probe backends on a schedule. Instead it watches real requests. Two parameters control this:

NGINX
upstream app_pool {
    least_conn;
    server 10.0.0.11:8080 max_fails=3 fail_timeout=30s;
    server 10.0.0.12:8080 max_fails=3 fail_timeout=30s;
}

If a server fails 3 times within 30 seconds, Nginx skips it for 30 seconds, then tries it again with a live request.

By default only connection errors and timeouts count as failures. To also retry on bad responses from a sick backend, set proxy_next_upstream in the location:

NGINX
location / {
    proxy_pass http://app_pool;
    proxy_next_upstream error timeout http_502 http_503 http_504;
    proxy_next_upstream_tries 2;
    proxy_connect_timeout 3s;
}

Active health checks (Nginx probing a URL on a timer) are part of the commercial NGINX Plus. With open source Nginx, use an external monitor to alert you when a backend goes down.

Keepalive to backends

Opening a new TCP connection to the backend for every request wastes time. Keep a pool of idle connections open:

NGINX
upstream app_pool {
    least_conn;
    server 10.0.0.11:8080 max_fails=3 fail_timeout=30s;
    server 10.0.0.12:8080 max_fails=3 fail_timeout=30s;
    keepalive 32;
}

And in the location:

NGINX
proxy_http_version 1.1;
proxy_set_header Connection "";

keepalive 32 is the number of idle connections each Nginx worker keeps per upstream, not a limit on total connections.

Apply the change

Terminal
sudo nginx -t && sudo systemctl reload nginx

On AlmaLinux or Rocky Linux, allow Nginx to open network connections if you have not already:

Terminal
sudo setsebool -P httpd_can_network_connect 1

Check it works

Add a header that shows which backend answered. Put this in the location temporarily:

NGINX
add_header X-Upstream $upstream_addr always;

Then send a few requests:

Terminal
for i in 1 2 3 4 5 6; do curl -sI https://example.com/ | grep -i x-upstream; done

You should see the backend addresses change. Stop one backend and repeat: after a few failures Nginx should only show the healthy ones. Check the error log for upstream ... failed lines, then remove the debug header.

Troubleshooting

  • Users get logged out at random: sessions are stored per backend. Move them to Redis or a database, or use ip_hash or a cookie hash.
  • One backend gets almost all traffic: you are using ip_hash behind a proxy, so every visitor has the same IP. Fix real IPs or switch methods.
  • 502 errors while one backend is down: failures are not retried. Add proxy_next_upstream and a short proxy_connect_timeout.
  • `nginx -t` says "invalid parameter backup": backup is not allowed with hash methods. Remove one or the other.
  • Backends overloaded after a restart: recovered servers rejoin at full weight at once. Stagger restarts and keep spare capacity.

Next steps

Was this guide helpful?

Still stuck?

Ask RunRacks AI for a quick answer, or open a ticket and our team will take a look at your server.

Websites & apps

Install PHP 8.x on AlmaLinux and Rocky Linux with Remi

Enable EPEL and the Remi repository on AlmaLinux or Rocky Linux 8 and 9, choose a PHP module stream, install PHP-FPM with common extensions, and run extra PHP versions side by side.

  • Intermediate
  • Linux VPS
  • 4 min read

Websites & apps

Use Apache as a reverse proxy with mod_proxy

Put Apache in front of a Node.js, Python or Docker app with mod_proxy. Set ProxyPass and ProxyPassReverse, keep the original host, proxy WebSockets and serve the app over HTTPS.

  • Intermediate
  • Linux VPS
  • 4 min read

Websites & apps

Enable gzip compression in Apache with mod_deflate

Compress HTML, CSS, JavaScript and JSON in Apache with mod_deflate, add Brotli with mod_brotli for modern browsers and check the Content-Encoding header to prove it works.

  • Beginner
  • Linux VPS
  • 4 min read