Nginx Reverse Proxy and Load Balancing¶

Intermediate45 minPrereqs: fundamentalsnginx reverse-proxy load-balancing websockets high-availability

Learning outcomes

Configure proxy_pass, forwarded headers, and buffering correctly for application backends
Distribute traffic across an upstream pool and choose the right load balancing algorithm
Handle backend failure with passive health checks, retries, and upstream keepalive connections

A reverse proxy accepts requests on behalf of servers that sit behind it. Nginx terminates TLS, serves static assets from disk, and forwards what remains to an application process that never talks to the internet directly. Add more than one of those application processes and the same configuration becomes a load balancer.

The Nginx Configuration guide introduces proxy_pass and upstream. This one covers what goes wrong in production: URI rewriting surprises, forwarded headers the backend cannot trust, buffering that breaks streaming responses, and failover behavior that quietly retries a payment twice.

What a Reverse Proxy Actually Does¶

Nginx does not forward the client's TCP connection. It terminates the client connection, builds a new HTTP request, and opens its own connection to the backend. Everything about the original request that the backend needs to know has to be copied across deliberately.

sequenceDiagram
    participant C as Client
    participant N as Nginx
    participant A as App server
    C->>N: TLS handshake, GET /api/orders
    N->>N: Match server block and location
    N->>N: Rebuild request, set proxy headers
    N->>A: Plain HTTP GET to upstream
    A->>N: 200 with response body
    N->>N: Buffer body, apply filters (gzip)
    N->>C: 200 over the original TLS connection

Two consequences follow from that rebuild, and most reverse proxy bugs trace back to one of them:

The backend sees Nginx's IP address as the client, and the connection as plain HTTP, unless you tell it otherwise.
The URI the backend receives is not necessarily the URI the client requested.

proxy_pass and the Trailing Slash¶

proxy_pass takes an address, optionally followed by a URI. Whether that URI is present changes how Nginx builds the request path, and the difference is a single character.

Without a URI part, the full client request URI is passed through unchanged:

location /api/ {
    proxy_pass http://127.0.0.1:8080;
}
# Client requests /api/orders  ->  backend receives /api/orders

With a URI part (even just /), Nginx replaces the part of the URI that matched the location with that URI:

location /api/ {
    proxy_pass http://127.0.0.1:8080/;
}
# Client requests /api/orders  ->  backend receives /orders

The replacement is literal string substitution, which produces doubled or missing slashes when the two sides disagree:

Location	proxy_pass	Client requests	Backend receives
`/api/`	`http://backend`	`/api/orders`	`/api/orders`
`/api/`	`http://backend/`	`/api/orders`	`/orders`
`/api/`	`http://backend/v2/`	`/api/orders`	`/v2/orders`
`/api`	`http://backend/`	`/api/orders`	`//orders`
`/api/`	`http://backend/v2`	`/api/orders`	`/v2orders`

Keep the trailing slashes matched on both sides or on neither side. The last two rows are the ones that reach production and produce a confusing 404 from the application framework rather than from Nginx.

proxy_pass takes no static URI part inside a regex location

A regex location has no matched prefix to substitute, so proxy_pass http://backend/v2/; inside location ~ ^/api/(.*) is a configuration error. The documented form is proxy_pass http://backend; with no URI at all, which forwards the request URI unchanged. When you need to rewrite it, build the URI from a capture: proxy_pass http://backend/$1;. The same restriction on static URI parts applies inside named locations, if blocks, and limit_except.

Variables in proxy_pass¶

Using a variable in the address changes Nginx's behavior in ways that are easy to miss. The address is evaluated per request rather than at startup, so when it resolves to a hostname Nginx needs a resolver directive to look it up, and a lookup failure produces a 502 at request time instead of a config error at reload. (A variable that holds a literal IP address or the name of an upstream block needs no resolver.)

resolver 127.0.0.11 valid=30s ipv6=off;   # Docker's embedded DNS

location /api/ {
    set $upstream_host api.internal;
    proxy_pass http://$upstream_host:8080;
}

The upside is that Nginx will pick up a changed DNS record without a reload, which matters in container environments where backend addresses move. The downside is that the URI is no longer normalized before being forwarded, and an unresolvable name takes the location down rather than failing the config test.

Forwarded Headers¶

By default the backend sees a request that appears to originate from Nginx over plain HTTP. Four headers fix that, and each has a failure mode.

location / {
    proxy_pass http://backend;

    proxy_set_header Host              $host;
    proxy_set_header X-Real-IP         $remote_addr;
    proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
}

Header	Value	Why
`Host`	`$host`	The hostname the client asked for. Frameworks use it to generate absolute URLs and to route multi-tenant requests. Without it the backend sees the upstream address.
`X-Real-IP`	`$remote_addr`	The immediate peer's address, as a single value.
`X-Forwarded-For`	`$proxy_add_x_forwarded_for`	Appends `$remote_addr` to any inbound `X-Forwarded-For`, building a chain.
`X-Forwarded-Proto`	`$scheme`	`http` or `https`. Without it a TLS-terminating proxy makes the app generate `http://` links and redirect loops.

$host and $http_host are not interchangeable. $http_host is the raw Host header, including any port and absent entirely if the client sent none. $host is the header lowercased with the port stripped, falling back to the matched server_name when there is no header at all. Prefer $host unless the backend genuinely needs the port.

X-Forwarded-For is attacker-controlled unless you strip it

$proxy_add_x_forwarded_for appends to whatever the client sent. A client that sends X-Forwarded-For: 127.0.0.1 gets that value preserved at the front of the chain, so a backend that reads the first entry as "the real client" can be trivially spoofed into believing a request came from localhost. If Nginx is your outermost proxy, overwrite instead of appending with proxy_set_header X-Forwarded-For $remote_addr;. If Nginx sits behind a CDN or another load balancer, use the realip module to define which upstream proxies you trust.

# Nginx sits behind a trusted load balancer at 10.0.0.0/8
set_real_ip_from 10.0.0.0/8;
real_ip_header X-Forwarded-For;
real_ip_recursive on;
# $remote_addr is now the real client, and the trusted hops are stripped

The Inheritance Trap¶

proxy_set_header is one of the array-valued directives that replaces rather than merges. Defining a single proxy_set_header inside a location discards every one inherited from the enclosing server or http block.

server {
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;

    location /api/ {
        proxy_pass http://backend;
        proxy_set_header X-Api-Version 2;
        # Host and X-Real-IP are NOT sent here - this one directive
        # replaced the whole inherited set
    }
}

Put the common headers in a snippet and include it in every location that adds its own:

# /etc/nginx/snippets/proxy-headers.conf
proxy_set_header Host              $host;
proxy_set_header X-Real-IP         $remote_addr;
proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;

location /api/ {
    proxy_pass http://backend;
    include snippets/proxy-headers.conf;
    proxy_set_header X-Api-Version 2;
}

Upstream Blocks¶

An upstream block names a pool of backends that proxy_pass can target by name.

upstream api_backend {
    zone api_backend 64k;
    least_conn;

    server 10.0.0.11:8080 weight=3 max_fails=2 fail_timeout=15s;
    server 10.0.0.12:8080 max_fails=2 fail_timeout=15s;
    server 10.0.0.13:8080 backup;

    keepalive 32;
}

Server Parameters¶

Parameter	Default	Effect
`weight=n`	`1`	Relative share of requests. `weight=3` receives three times the traffic of a `weight=1` peer.
`max_fails=n`	`1`	Failed attempts within `fail_timeout` before the server is considered unavailable. `max_fails=0` disables the check entirely.
`fail_timeout=time`	`10s`	Both the window in which failures are counted and how long the server stays out of rotation.
`backup`	off	Receives traffic only when every non-backup server is unavailable.
`down`	off	Marks a server as permanently unavailable. Useful for draining a node before maintenance.
`max_conns=n`	`0` (unlimited)	Caps simultaneous active connections to that server.

The zone Directive¶

Without zone, each worker process keeps its own private copy of the load balancing state and failure counters. With eight workers, a backend has to fail max_fails times per worker before it is fully out of rotation, and least_conn balances against connection counts that only one worker can see.

zone puts that state in shared memory so all workers agree. It has been available in open source Nginx since 1.9.0 and costs a few kilobytes. Use it on any upstream with more than one server.

Load Balancing Algorithms¶

upstream backend {
    least_conn;                 # the method goes before the server list
    server 10.0.0.11:8080;
    server 10.0.0.12:8080;
}

Method	Directive	Behavior and when to use it
Round robin	(default, no directive)	Requests distributed in order, respecting `weight`. Correct default when backends are identical and requests cost about the same.
Least connections	`least_conn`	Sends to the peer with the fewest active connections. Better when request durations vary widely, for example a mix of fast reads and slow report generation.
IP hash	`ip_hash`	Hashes the client address so a given client always reaches the same peer. Session persistence without shared session storage.
Generic hash	`hash $request_uri consistent`	Hashes an arbitrary key. With `consistent`, uses ketama hashing so adding a peer remaps only a fraction of keys. Good for cache-node affinity.
Random	`random two least_conn`	Picks two peers at random and sends to the better of the two. Behaves well across multiple load balancers that cannot see each other's connection counts.

ip_hash uses the first three octets of an IPv4 address (so a /24 of clients lands on one peer) and the whole address for IPv6. Two caveats matter in practice: adding or removing a server rehashes most clients and scatters their sessions, and every client behind one corporate NAT gateway lands on the same backend. Real session storage in Redis or the database is a better answer than IP affinity wherever you can manage it.

Several familiar directives are commercial-only

sticky cookie persistence, least_time, queue, slow_start, and the active health_check directive are NGINX Plus features. They will fail nginx -t with unknown directive on open source builds. Everything else in this guide works on the open source release.

Health Checks and Failover¶

Open source Nginx does passive health checking. It does not poll backends on a timer; it observes real requests and takes a peer out of rotation after max_fails failures within fail_timeout.

upstream backend {
    zone backend 64k;
    server 10.0.0.11:8080 max_fails=3 fail_timeout=30s;
    server 10.0.0.12:8080 max_fails=3 fail_timeout=30s;
}

After 3 failures inside 30 seconds, that peer is skipped for 30 seconds, then a single request is allowed through to probe it. If that request succeeds the peer returns to the pool; if it fails the timer restarts.

What counts as a failure is defined by proxy_next_upstream, which does double duty: it decides both when to retry the current request on another peer and what increments the failure counter.

location / {
    proxy_pass http://backend;
    proxy_next_upstream error timeout http_502 http_503 http_504;
    proxy_next_upstream_tries 2;
    proxy_next_upstream_timeout 10s;
}

Value	Meaning
`error`	Connection failure, or a failure while sending the request or reading the response header. Default.
`timeout`	A connect, send, or read timeout was reached. Default.
`invalid_header`	The backend returned a malformed or empty response.
`http_502` / `http_503` / `http_504`	Treat those status codes as failures worth retrying.
`http_500` / `http_403` / `http_404`	Retry on these too. Rarely a good idea - a 404 is usually a correct answer, not a sick backend.
`non_idempotent`	Also retry POST, LOCK, and PATCH. Off by default, deliberately.
`off`	Never retry.

Do not add non_idempotent without an idempotency key

Retries default to GET, HEAD, PUT, DELETE, and OPTIONS because those are safe to repeat. Adding non_idempotent lets Nginx replay a POST on a second backend when the first times out - and a timeout does not mean the first backend failed to process it. The classic outcome is a duplicate charge or a duplicate order. Only enable it if the backend deduplicates on an idempotency key.

Always bound the retries. proxy_next_upstream_tries 0 (the default) means "try every peer in the pool", so one slow request can burn proxy_connect_timeout against every backend in turn before answering the client.

Upstream Keepalive¶

By default Nginx opens a new TCP connection to the backend for every request and closes it afterwards. Under load that is a measurable amount of handshake overhead and a steady supply of sockets in TIME_WAIT.

The keepalive directive maintains a cache of idle connections per worker. It needs two companion directives in the location, and omitting them silently disables the benefit.

upstream backend {
    zone backend 64k;
    server 10.0.0.11:8080;
    server 10.0.0.12:8080;

    keepalive 32;              # idle connections cached per worker
    keepalive_timeout 60s;     # how long an idle connection is kept
    keepalive_requests 1000;   # requests before the connection is recycled
}

server {
    location / {
        proxy_pass http://backend;
        proxy_http_version 1.1;        # keepalive requires HTTP/1.1
        proxy_set_header Connection "";  # clear the inherited "close"
        include snippets/proxy-headers.conf;
    }
}

proxy_http_version 1.1; matters because HTTP/1.0 has no persistent connections. Nginx used 1.0 for proxying by default until 1.29.7, which changed the default to 1.1. Distribution packages lag mainline by a long way, so set it explicitly rather than assuming the build in front of you has the newer default. proxy_set_header Connection ""; is required because Nginx otherwise forwards a Connection: close header that tells the backend to hang up after one response.

The keepalive number is idle connections per worker, not a total or a limit on concurrency. With 8 workers and keepalive 32 you may hold up to 256 idle upstream connections, so the backend's own connection limit needs headroom above that.

WebSocket and Streaming Proxying¶

A WebSocket handshake is an HTTP request carrying Upgrade: websocket and Connection: Upgrade. Nginx will not forward hop-by-hop headers unless told to, so the upgrade has to be reconstructed.

The naive version hardcodes the header:

proxy_set_header Connection "upgrade";   # breaks plain requests

That sends Connection: upgrade on every request through the location, including ordinary REST calls that never asked to upgrade, which confuses some backends and defeats upstream keepalive. The correct pattern maps the header so it is only set when the client actually requested an upgrade:

# http context, defined once
map $http_upgrade $connection_upgrade {
    default upgrade;
    ''      close;
}

server {
    location /ws/ {
        proxy_pass http://backend;

        proxy_http_version 1.1;
        proxy_set_header Upgrade    $http_upgrade;
        proxy_set_header Connection $connection_upgrade;
        include snippets/proxy-headers.conf;

        proxy_read_timeout 3600s;   # a quiet socket is not a dead socket
        proxy_send_timeout 3600s;
    }
}

proxy_read_timeout defaults to 60 seconds and applies between reads, not to the connection's total lifetime. A WebSocket that sits idle for longer than that is closed by Nginx, which the client experiences as a random disconnect. Either raise the timeout or have the application send periodic pings.

Server-Sent Events and Streaming¶

Nginx buffers responses by default: it reads the backend's response as fast as the backend can produce it, stores it, and feeds it to the client at whatever rate the client can accept. That protects the backend from slow clients and is the right default for normal responses.

It is exactly wrong for streaming. With buffering on, a server-sent events endpoint or a streamed LLM response accumulates in Nginx's buffers and arrives at the client in chunks, or not until the response ends.

location /events/ {
    proxy_pass http://backend;
    proxy_http_version 1.1;
    proxy_set_header Connection "";

    proxy_buffering off;        # forward each chunk as it arrives
    proxy_cache off;
    proxy_read_timeout 24h;
}

Directive	Default	What to know
`proxy_buffering`	`on`	Off means chunks stream through immediately, at the cost of holding a backend connection open for slow clients.
`proxy_buffer_size`	`4k` or `8k`	Buffer for the response header only. Raise it when a backend sends large cookies and you see `upstream sent too big header`.
`proxy_buffers`	`8 4k` or `8 8k`	Count and size of body buffers per connection.
`proxy_busy_buffers_size`	`8k` or `16k`	How much buffered data can be sent to the client while the rest is still being read.
`proxy_max_temp_file_size`	`1024m`	Responses larger than the buffers spill to disk. Set to `0` to disable spooling and force streaming.

A backend can also opt out per response by sending X-Accel-Buffering: no, which is the cleanest option when only some endpoints stream.

Debugging a Proxy¶

The upstream variables belong in your log format from day one. Without them you cannot distinguish a slow backend from a slow client, or see that a request was retried.

log_format upstream_log '$remote_addr - [$time_local] "$request" '
                        '$status $body_bytes_sent '
                        'upstream=$upstream_addr '
                        'up_status=$upstream_status '
                        'rt=$request_time uct=$upstream_connect_time '
                        'urt=$upstream_response_time';

access_log /var/log/nginx/access.log upstream_log;

When a request is retried, $upstream_addr and $upstream_status contain a comma-separated list of every peer tried, which makes failover visible in the log:

upstream=10.0.0.11:8080, 10.0.0.12:8080 up_status=502, 200 rt=0.243 urt=0.019, 0.201

The status codes Nginx generates itself are the fastest way to narrow a problem:

Code	Meaning	Usual cause
`502 Bad Gateway`	Nginx reached the backend but the response was unusable	Backend crashed, refused the connection, or sent a malformed response
`504 Gateway Timeout`	The backend accepted the connection but did not answer in time	`proxy_read_timeout` exceeded; a slow query or a deadlocked worker
`499`	Client closed the connection before Nginx answered	The user navigated away, or a client-side timeout is shorter than the backend's response time
`413 Request Entity Too Large`	Body exceeded `client_max_body_size`	The 1 MB default blocks most file uploads
`upstream sent too big header`	Response header exceeded `proxy_buffer_size`	Large cookies or a long redirect chain in the header

A wave of 499s is worth taking seriously: it means clients are giving up, so the real problem is latency, not the disconnects themselves.

Putting It All Together¶

Interactive Quizzes¶

You add `proxy_set_header X-Request-Id $request_id;` inside a location that previously inherited Host and X-Forwarded-For from the server block. What happens? (requires JavaScript)