Upstream and Load Balancing

Introduction

When one app instance is not enough—or you run two copies for zero-downtime deploys—Nginx load balances across multiple upstream servers. This chapter defines an upstream block, uses round-robin and weights, and explains passive health behavior in open-source Nginx.

Prerequisites

• Proxying Spring Boot and Node.js
• Proxy Headers and Path Rules
• Two app instances on different ports for experiments (optional)

upstream Block

Code explanation:

• upstream backend names a group of servers
• proxy_pass http://backend uses the group (note: no trailing slash on name)
• Default method: round-robin—requests alternate between 8080 and 8081

Reload after edit:

sudo nginx -t && sudo systemctl reload nginx

Test Round-Robin Locally

Run two simple listeners (example with Node):

# Terminal 1
node -e "require('http').createServer((q,s)=>s.end('8080')).listen(8080,'127.0.0.1')"
 
# Terminal 2
node -e "require('http').createServer((q,s)=>s.end('8081')).listen(8081,'127.0.0.1')"

for i in 1 2 3 4; do curl -s -H "Host: api.example.com" http://127.0.0.1/; echo; done

You should see responses alternate between 8080 and 8081.

Weighted Servers

Send more traffic to a stronger machine:

upstream backend {
    server 127.0.0.1:8080 weight=3;
    server 127.0.0.1:8081 weight=1;
}

Code explanation:

• weight=3 gets three shares per one share of weight=1
• Useful when instances differ in CPU/RAM

max_fails and fail_timeout

Temporarily mark server down after errors:

upstream backend {
    server 127.0.0.1:8080 max_fails=3 fail_timeout=30s;
    server 127.0.0.1:8081 max_fails=3 fail_timeout=30s;
}

Code explanation:

• max_fails — failures within fail_timeout window before marking down
• fail_timeout — how long to avoid the server after marking down
• Passive—based on actual proxy errors, not active HTTP health probes

backup Server

upstream backend {
    server 127.0.0.1:8080;
    server 127.0.0.1:8081;
    server 127.0.0.1:8082 backup;
}

backup receives traffic only when primary servers are unavailable.

ip_hash (Session Stickiness)

upstream backend {
    ip_hash;
    server 127.0.0.1:8080;
    server 127.0.0.1:8081;
}

Same client IP tends to hit the same backend—helps legacy sessions stored in memory. Prefer stateless JWT or shared session store in new apps.

Keepalive to Upstream

Reduce connection overhead:

Advanced optimization—defaults work for learning setups.

When Load Balancing Helps

On a single small VPS, one instance plus Nginx is enough until CPU or connections saturate.

Limitations (Open Source)

• No built-in active health check HTTP GET to /health (without third-party modules)
• All backends must handle same routes unless using split locations
• Sticky sessions via ip_hash breaks if client IP changes (mobile networks)

FAQ

upstream name vs URL?

proxy_pass http://backend uses named group. proxy_pass http://127.0.0.1:8080 is single server—no upstream block.

Can I mix Spring and Node in one upstream?

Technically yes if responses are compatible—almost never done. Split by path or subdomain instead.

Load balance to remote servers?

server 10.0.0.10:8080;
server 10.0.0.11:8080;

Private network IPs common in cloud VPCs.

502 when one server down?

With two servers, Nginx should skip failed one after max_fails. With one server, any down = 502.

Kubernetes replaces Nginx LB?

K8s Services/Ingress often replace manual upstream—Nginx skills still apply to Ingress controllers.

least_conn method?

upstream backend {
    least_conn;
    server ...;
}

Sends to server with fewest active connections—good for long-lived requests.