Load balancing

Load balancers and ingress at L3, L4 and L7.

You put these in front of your processes and VPC members. A LoadBalancer spreads traffic over backends and checks their health. An Ingress is where traffic comes in, and it hands it to backends, to a load balancer, or to a Database or MessageBroker. For example, an Ingress can end TLS for your API's hostname and pass the requests to a LoadBalancer with four backend processes behind it.

The three resources

A LoadBalancer spreads traffic over backends on one VPC. An Ingress is the entry point that accepts traffic and sends it on to its targets. A Certificate gives an ingress listener its TLS certificate. Each load balancer and each ingress has 1 to 5 nodes on its VPC.

LoadBalancer

Works at l3 (IP packets), l4 (TCP and UDP) or l7 (HTTP/1.1). Round robin, least connections or hashing, with health checks.

Ingress

The way in. At l7 it routes by host, path and header, and it can end TLS 1.2 or 1.3.

Certificate

A generated self-signed certificate, or a certificate and key you keep in a bucket of your project.

IPv6 first

A load balancer works over IPv6 alone. IPv4 backends need an IPv4 network on the VPC.

You pick a level when you create one, and the level decides what a node looks at.

LevelLoadBalancerIngress
l3Forwards IP packets. The backend is chosen per flow by hash, so a flow does not move.Forwards every IP packet addressed to it to one target.
l4TCP and UDP listeners. One connection, or one client address and port, goes to one backend.TCP and UDP listeners, each with its own target.
l7HTTP/1.1 listeners. Adds X-Forwarded-For, X-Forwarded-Proto and X-Request-Id. TLS is ended by an Ingress, not here.HTTP and HTTPS listeners, with rules by host, path and header and a default target.

A backend can be a Compute process, a VPC member or a plain address. There is more in the load balancing guide.

How a backend is chosen

A load balancer picks a backend by weighted round robin, by least connections divided by weight, or by rendezvous hashing on the client address, the flow, a header or a cookie. Hash on a cookie when a user's session should keep landing on the same backend. With hashing, removing a backend moves only the keys that were on that backend, and everyone else stays where they were.

Weights go from 0 to 256 and default to 1. Setting a backend's weight to 0 drains it. It gets no new flows and the ones it has carry on, which is the way to take a backend out for maintenance.

A backend starts as Unknown. It becomes Up once it has passed the healthy threshold of checks, and Down once it has failed the unhealthy threshold. While every backend is still Unknown, traffic goes to them as if all were up.

Routing HTTP

At l7 a request goes to the first rule whose host, path and headers all match. If none matches it goes to the default target, and if there is no default it gets a 404. A path prefix matches on a segment boundary, so /api matches /api/x but not /apix.

The node strips hop-by-hop headers. It refuses a request that carries both Content-Length and Transfer-Encoding, or two different lengths, because those are the requests that let a second request hide inside the first one on its way past a proxy.

A kept-alive HTTP connection is routed by its first request. If one client sends requests for two different hosts over the same connection, they all go where the first one went.

Certificates and private keys

A private key is never part of an API object. A generated key stays in the node's memory. A key you supply is an object in your bucket, and the node reads it when it starts. Keys never show up in a log line or an error message, and nodes report only a certificate's fingerprint and expiry.

Generated certificates are self-signed, made by each node and valid for 90 days. Clients do not trust them, so for anything a browser or an outside client will see, store your own certificate and key in a bucket.

How the nodes work

Connection, flow and TLS state lives in each node's memory, not in storage. Nodes share nothing. Any node accepts any flow, and hash selection gives the same answer on every node, so they agree on where a flow belongs without talking to each other. After a restart, hash selection sends flows to the same backend again.

Keeping state in memory has consequences you should plan for. A node that restarts starts empty, so its clients reconnect and a generated TLS key is made again. Connection and flow counts are per node. Each node makes its own self-signed certificate. And a client reaches a particular node through DNS or through an Ingress.

A node reads its plan when it starts. Health checks decide which backends get traffic but do not change the plan, so a backend that keeps flapping never rewrites configuration. Changing the plan yourself replaces the node, which ends its connections.

The status of each object lists its node addresses, IPv6 first, and the state of every backend or target.

Limits

PropertyValue
Backends1 to 64 per load balancer. Weight 0 to 256, default 1; weight 0 drains a backend.
Nodes1 to 5 per load balancer or ingress, default 1. Nodes share no state.
Node memory64 KiB × 2 to the power memoryPower, 0 to 10, default 6 (4 MiB).
Listeners and rulesUp to 8 listeners per load balancer or ingress, and up to 32 rules per ingress.
Fixed at creationThe VPC and the level.
HTTPHTTP/1.1 and WebSocket only. A request head is at most 16 KiB and 100 headers. A kept-alive HTTP connection is routed by its first request.
TLS1.2 or 1.3, with HTTP/1.1 only. No client certificates.
Certificates1 to 16 domain names. Self-signed, made by each node and valid 90 days, or stored by you in a bucket. Clients do not trust a self-signed certificate. A node reads its certificate when it starts.
Health checkstcp, http or icmp. Interval 1 to 300 s (default 5), timeout 1 to 60 s (default 2), thresholds 1 to 10.
l3 forwardingIPv6 only. No extension headers or fragments, and ICMPv6 errors are dropped. A flow idles out after 30 s.
TCP relay48 connections on the smallest node.
Addresses and targetsOn the VPC.

When things fail

What failsWhat happens
A backendAfter the configured number of failed checks it is taken out of selection, and flows to it end. Hash selection moves only that backend's keys.
Every backendNew flows are refused, and l7 answers 503. Traffic is never sent to a backend that is down.
A nodeIts address stays in status.addresses, the phase becomes Degraded, and clients retry another address. A node that restarts starts empty: clients reconnect, and a generated TLS key is made again.
A target that was deletedIt is reported Down with reason NotFound.
A malformed request headA head over 16 KiB or with more than 100 headers gets 400 or 431, and the connection is closed.

You can manage them with kubectl or in the console. The quickstart is the place to begin. For where load balancers sit in the network, see the architecture page.

Request access