Server Sent Events In a Kubernetes Cluster

Server Sent Events

Kubernetes

SSE

Cloud Infrastructure

Microservices

Server Sent Events In a Kubernetes Cluster

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design

Kubernetes AWS & Cloud Distributed Systems DevOps & Deployment

Introduction

Server-Sent Events (SSE) enable a server to push real-time updates to clients over a single long-lived HTTP connection. Running SSE in Kubernetes introduces challenges around connection persistence, load balancer timeouts, pod scaling, and graceful shutdowns. Unlike WebSockets, SSE uses plain HTTP, which simplifies infrastructure but requires careful configuration of ingress controllers, timeouts, and buffering to prevent connections from being silently dropped.

How SSE Works

1Client                          Server
2  |--- GET /events (Accept: text/event-stream) --->|
3  |<--- HTTP 200 (Content-Type: text/event-stream) |
4  |<--- data: {"temp": 72}\n\n                     |
5  |<--- data: {"temp": 73}\n\n                     |
6  |<--- data: {"temp": 71}\n\n                     |
7  |             ... connection stays open ...        |

The client opens a single HTTP request, and the server sends events as they occur. The connection remains open indefinitely. If it drops, the browser's EventSource API automatically reconnects.

Basic SSE Server (Node.js)

javascript

1const express = require('express');
2const app = express();
3
4app.get('/events', (req, res) => {
5    res.writeHead(200, {
6        'Content-Type': 'text/event-stream',
7        'Cache-Control': 'no-cache',
8        'Connection': 'keep-alive',
9        'X-Accel-Buffering': 'no'  // Disable nginx buffering
10    });
11
12    const interval = setInterval(() => {
13        const data = JSON.stringify({ time: new Date().toISOString() });
14        res.write(`data: ${data}\n\n`);
15    }, 1000);
16
17    req.on('close', () => {
18        clearInterval(interval);
19        res.end();
20    });
21});
22
23app.listen(3000);

Kubernetes Deployment

yaml

1apiVersion: apps/v1
2kind: Deployment
3metadata:
4  name: sse-server
5spec:
6  replicas: 3
7  selector:
8    matchLabels:
9      app: sse-server
10  template:
11    metadata:
12      labels:
13        app: sse-server
14    spec:
15      containers:
16        - name: sse-server
17          image: my-sse-server:latest
18          ports:
19            - containerPort: 3000
20          readinessProbe:
21            httpGet:
22              path: /health
23              port: 3000
24          livenessProbe:
25            httpGet:
26              path: /health
27              port: 3000
28      terminationGracePeriodSeconds: 60
29---
30apiVersion: v1
31kind: Service
32metadata:
33  name: sse-service
34spec:
35  selector:
36    app: sse-server
37  ports:
38    - port: 80
39      targetPort: 3000

Ingress Configuration (NGINX)

The most critical piece — NGINX ingress must not timeout or buffer SSE connections:

yaml

1apiVersion: networking.k8s.io/v1
2kind: Ingress
3metadata:
4  name: sse-ingress
5  annotations:
6    nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
7    nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
8    nginx.ingress.kubernetes.io/proxy-buffering: "off"
9    nginx.ingress.kubernetes.io/proxy-cache: "off"
10    nginx.ingress.kubernetes.io/connection-proxy-header: "keep-alive"
11spec:
12  rules:
13    - host: sse.example.com
14      http:
15        paths:
16          - path: /events
17            pathType: Prefix
18            backend:
19              service:
20                name: sse-service
21                port:
22                  number: 80

Key annotations:

proxy-read-timeout: "3600" — keep connections open for 1 hour (default is 60s)
proxy-buffering: "off" — send events immediately without buffering
proxy-cache: "off" — do not cache event streams

Session Affinity (Sticky Sessions)

SSE connections are stateful — a client should reconnect to the same pod if possible:

yaml

1apiVersion: v1
2kind: Service
3metadata:
4  name: sse-service
5  annotations:
6    service.beta.kubernetes.io/aws-load-balancer-stickiness-enabled: "true"
7spec:
8  sessionAffinity: ClientIP
9  sessionAffinityConfig:
10    clientIP:
11      timeoutSeconds: 3600
12  selector:
13    app: sse-server
14  ports:
15    - port: 80
16      targetPort: 3000

Without sticky sessions, a reconnecting client may hit a different pod that does not have its subscription state.

Graceful Shutdown

When a pod is terminated (scaling down, rolling update), active SSE connections must be closed cleanly:

javascript

1process.on('SIGTERM', () => {
2    console.log('SIGTERM received, closing SSE connections');
3
4    // Stop accepting new connections
5    server.close();
6
7    // Send a close event to all connected clients
8    activeClients.forEach(client => {
9        client.res.write('event: close\ndata: server shutting down\n\n');
10        client.res.end();
11    });
12
13    // Exit after giving clients time to reconnect to another pod
14    setTimeout(() => process.exit(0), 5000);
15});

Set terminationGracePeriodSeconds in the pod spec to give the shutdown handler enough time.

Scaling Considerations

With multiple pods, each pod only knows about its own connected clients. To broadcast events to all clients across pods, use a message broker:

                    ┌─── Pod 1 (50 clients)
Redis Pub/Sub ──────┼─── Pod 2 (50 clients)
                    └─── Pod 3 (50 clients)

javascript

1const Redis = require('ioredis');
2const subscriber = new Redis();
3const publisher = new Redis();
4
5// Each pod subscribes to the channel
6subscriber.subscribe('events');
7subscriber.on('message', (channel, message) => {
8    // Broadcast to all locally connected clients
9    activeClients.forEach(client => {
10        client.res.write(`data: ${message}\n\n`);
11    });
12});
13
14// Publishing an event reaches all pods
15publisher.publish('events', JSON.stringify({ type: 'update', data: 'new data' }));

Horizontal Pod Autoscaler

yaml

1apiVersion: autoscaling/v2
2kind: HorizontalPodAutoscaler
3metadata:
4  name: sse-hpa
5spec:
6  scaleTargetRef:
7    apiVersion: apps/v1
8    kind: Deployment
9    name: sse-server
10  minReplicas: 2
11  maxReplicas: 10
12  metrics:
13    - type: Resource
14      resource:
15        name: cpu
16        target:
17          type: Utilization
18          averageUtilization: 70
19    - type: Pods
20      pods:
21        metric:
22          name: active_connections
23        target:
24          type: AverageValue
25          averageValue: "1000"

Scale based on connection count rather than CPU, since SSE connections are mostly idle but consume memory and file descriptors.

Common Pitfalls

Default NGINX timeout kills connections: The default proxy-read-timeout is 60 seconds. SSE connections that go 60 seconds without an event are terminated. Set it to 3600 or higher, and send periodic heartbeat events (:heartbeat\n\n) to keep the connection alive.
Proxy buffering delays events: NGINX and cloud load balancers buffer responses by default. A buffered SSE event is not sent until the buffer fills, causing delayed or batched delivery. Set proxy-buffering: off and X-Accel-Buffering: no.
Pod termination drops connections without notice: Without a graceful shutdown handler, Kubernetes kills the pod and clients see a network error. Send a close event before shutting down and set an adequate terminationGracePeriodSeconds.
No cross-pod broadcasting: Each pod only pushes events to its own clients. Without Redis Pub/Sub or a similar message bus, events published on one pod are invisible to clients connected to other pods.
File descriptor limits: Each SSE connection uses a file descriptor. The default ulimit may be too low for thousands of connections. Set ulimit -n 65535 in the container or use securityContext.rlimits in the pod spec.

Summary

SSE uses long-lived HTTP connections — configure ingress timeouts to 3600+ seconds
Disable proxy buffering (proxy-buffering: off) to deliver events immediately
Use session affinity (sticky sessions) so reconnecting clients hit the same pod
Implement graceful shutdown to close SSE connections cleanly during pod termination
Use Redis Pub/Sub or a message broker to broadcast events across multiple pods
Scale based on connection count rather than CPU for SSE workloads

System Design Fundamentals

Build a strong foundation in designing scalable, reliable distributed systems.

View the course

Track what you have practised

A free account saves your progress, solutions and study plan across every problem on Codemia.

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design

Server Sent Events In a Kubernetes Cluster

System Design practice on Codemia

Introduction

How SSE Works

Basic SSE Server (Node.js)

Kubernetes Deployment

Ingress Configuration (NGINX)

Session Affinity (Sticky Sessions)

Graceful Shutdown

Scaling Considerations

Horizontal Pod Autoscaler

Common Pitfalls

Summary

Related reading

System Design Fundamentals

Track what you have practised

System Design practice on Codemia