High-level design

This section includes a description of the components that implement the system for fulfilling the requirements.

It's complemented with the C4 Model system context and container diagrams.

System Context

A Kubernetes cluster is used to run most of the components.

Edge layer

Includes components that handle system incoming requests.

Load balancer

A load balancer ensures load distribution and supports high availability sending requests to active nodes.

A TCP load balancer (L4) is used because no complex routing logic is needed. Once the WebSocket connection has been established, it is maintained as a persistent connection. If the connection fails, due to a communication problem or one of the component fails (API Gateway pod, online presence microservice), a new connection will be established by the client application.

API Gateway

An API Gateway is used to:

Rate limit requests
Route new sessions to the presence microservice
Centralized logging
Check session authentication
Message max length validation avoiding big problematic fraudulent requests

APISIX is the selected one because it's low latency, high throughput, high availability, stateless and extensible architecture. It is an Apache open source top project with good community and support. It will be implemented as a Kubernetes Ingress Controller.

Other good alternatives like Kong and NGINX were considered and APISIX selected because its features:

Better performance and lower latency
High availability and scalability (stateless data plane) by design
Lower TCO
Cloud-native solution
Full dynamic configuration

Kong has a stronger plugin ecosystem. APISIX, as a top‑level Apache Software Foundation project, is growing rapidly.

Application layer

Contains the core applications and background workers that provide the business logic. All backend components will be implemented with Node.js, chosen for its scalable, lightweight, and efficient performance in I/O-bound, real-time applications.

An alternative like SpringBoot was considered but discarded due to its higher resource footprint and slower scaling for this specific use case.

Presence management

Container Presence Management

ComponentResponsibility
Load Balancer (L4)	Routes WSS and IAM traffic to healthy nodes; performs TCP health checks and draining.
API Gateway (APISIX)	Terminates WSS, rate limits, validates JWT (JWKS/introspection), forwards to service.
IAM (Keycloak)	Authenticates users, issues tokens, exposes JWKS and introspection APIs.
Presence Microservice	Manages sessions and subscriptions; publishes updates; pushes notifications over WSS.
Database (Dragonfly)	Stores user status and subscription mappings.
Messaging Cluster (Dragonfly Pub/Sub)	Low-latency fan-out channel for presence updates.

Identity and access management

Provides authentication in a specific, specialized secure solution. Keycloak will be used to authenticate users and issue JWTs for establishing the WebSocket connection.

Keycloak is selected because it's an efficient, popular solution that helps to avoid lock-in. Proprietary features will be avoided, sticking to standards.

Other alternatives considered were Auth0/Okta and AWS Cognito. These SaaS solutions provide lower TCO but higher lock‑in and were discarded.

Online presence microservice

This service's primary role is to manage the persistent WebSocket connection with clients. Its responsibilities are:

Handling subscription commands (subscriptions:*) to modify user data.
Acting as a producer of activity events (pings) to the session management system.
Publishing and subscribing to the Pub/Sub broker to fan-out real-time presence updates.

The service will be implemented with Node.js because it provides a very scalable, lightweight solution ideal for I/O-bound real-time applications.

Spring Boot has been discarded because it is heavier for a real‑time application and slower to scale when needed.

Session lifecycle

Container Session Lifecycle

ComponentResponsibility
Load Balancer (L4)	Routes WSS traffic from clients to the API Gateway.
API Gateway (APISIX)	Forwards authenticated WSS traffic; enforces coarse limits.
Presence Microservice	Receives pings and writes them to `pings:{shard}` stream.
Ping Consumers	Consume `pings:{shard}`, update last seen, and reschedule deadlines.
Scheduler	Finds expired deadlines; emits `inactive` events to `presence_events:{shard}`.
Cleanup Worker	Consumes `inactive` events and deletes the user's presence/session data.
Database (Dragonfly)	Holds `h:last_seen:{shard}` and `z:deadlines:{shard}` indexes.
Messaging Cluster (Streams)	Hosts durable `pings` and `presence_events` streams.

Ping Consumers (Consumer Group)

A set of background workers that reliably process the activity event stream. They are responsible for updating the last seen status for each user and rescheduling their inactivity deadline. These will be implemented as a standard Kubernetes Deployment.

Scheduler

A background worker that periodically checks for users whose inactivity deadlines have passed. After verification, it publishes an "inactive" event to trigger the final cleanup. This will be implemented as a Kubernetes CronJob to ensure periodic execution.

Cleanup Worker (Event-Driven)

A background worker that listens for "inactive" events. Upon receiving an event, it performs the full, permanent cleanup of all data associated with the stale user. This will be implemented as a standard Kubernetes Deployment.

Database

A dedicated Dragonfly cluster that stores all core presence and session lifecycle data. Its in-memory, multi-threaded architecture was chosen for its low latency and high throughput, which are critical for this real-time application.

Pub/Sub Broker

A separate, dedicated Dragonfly cluster used for the real-time, low-latency fan-out of presence updates. This isolates the critical messaging fabric from the database, ensuring high availability.

Using Dragonfly for both roles simplifies the technology stack and reduces operational overhead, which was preferred over introducing a different system like RabbitMQ or Kafka.

Event Streaming

For reliable session management, we will use Dragonfly Streams. This feature provides a persistent, append-only log for processing activity events (pings) with at-least-once delivery guarantees.

The streams will be hosted on the same Dragonfly cluster as the Pub/Sub broker, creating a unified "Messaging Cluster" that is isolated from the primary database.

Analytics, Monitoring & Alerting

This layer collects and provides useful information for business analysts and SREs to monitor system health and usage.

Container Observability

ComponentResponsibility
Grafana	Unified dashboards for metrics, logs, and traces; SLO visualization.
Prometheus	Scrapes, stores, and queries metrics; feeds Alertmanager.
Alertmanager	Deduplicates, groups, and routes alerts to on-call.
Loki	Ingests and indexes logs; queried by Grafana.
Tempo	Stores distributed traces; queried by Grafana.
OpenTelemetry Collector	Receives OTLP telemetry; forwards traces and metrics (e.g., from CronJobs).
Fluent Bit	Collects pod logs and forwards to Loki.

Observability Stack: The Grafana LGTM Stack

For our observability needs, we have selected the Grafana "LGTM" (Loki, Grafana, Tempo, Mimir/Prometheus) stack. As the company has a mature platform engineering team, this stack provides a cloud-native, highly available, and scalable solution that avoids vendor lock-in. It offers an efficient TCO at scale and provides a single pane of glass for metrics, logs, and traces, which is crucial for rapid troubleshooting.

OpenSearch stack has also been considered

A key metric to monitor will be the consumer lag for the pings stream (XPENDING).

Components

Visualization (Grafana): The central dashboarding platform for both business and technical metrics.
Metrics (Prometheus): Scrapes and stores time-series data for monitoring and analysis.
Alerting (Alertmanager): Manages alert deduplication, grouping, and routing based on Prometheus data.
Telemetry Collector (OpenTelemetry Collector): Collects application metrics for ephemeral application components (Scheduler).

Logging

This layer is responsible for the collection, storage, and querying of application logs and traces to support troubleshooting and debugging by SRE and DevOps teams.

Components

We will leverage components from our chosen Shared Observability Stack:

Visualization (Grafana): Used to search, query, and correlate logs and traces.
Log Storage (Loki): Stores and indexes application logs.
Trace Storage (Tempo): Stores and retrieves distributed traces.
Log Collector (Fluent Bit): A lightweight agent to collect logs and forward them to Loki.
Telemetry Collector (OpenTelemetry Collector): Collects application traces and forwards them to Tempo.

Observability & SLOs

SLOs: P99 publish→client push latency ≤ 500 ms; availability ≥ 99.9%; missed-update rate ≤ 0.1%/day.
Metrics: WebSocket connection count, send-queue depth, broker publish rate, pub/sub end-to-end latency, stream XPENDING lag, consumer throughput, error rates.
Alerts: on sustained P99 latency breaches, rising XPENDING, consumer restart storms, and publish failures.

High Availability

This section summarizes how HA is achieved for each component. The baseline target is ≥99.9% service availability.

Kubernetes cluster

Two availability zones (AZ-A, AZ-B)
Managed HA control plane (or 3 dedicated control plane nodes)
Worker nodes distributed across AZs
Reliability measures:
- PodDisruptionBudgets (PDBs)
- Topology spread constraints
- Anti-affinity to avoid colocation
Updates and scaling:
- Rolling updates with surge
- Horizontal Pod Autoscaler (HPA)
- Cluster Autoscaler enabled

L4 load balancer

Multi-AZ nodes
TCP health checks to API Gateway
Connection draining during rollouts
Routes:
- WebSocket (WSS) traffic
- IAM (Keycloak) traffic

API Gateway (APISIX)

3+ replicas, stateless data plane
APISIX Ingress Controller: 2 replicas
Config store (etcd):
- HA quorum of 3 nodes across AZs
Resilience:
- Readiness/liveness probes
- PodDisruptionBudgets (PDBs)
- PreStop hooks for WebSocket connection draining

Presence microservice

3+ replicas, stateless
Autoscaling:
- HPA on CPU usage
- HPA on custom metrics (active sessions, send-queue depth)
Reliability measures:
- Readiness/liveness probes
- PDBs
- Topology spread constraints
Graceful termination:
- PreStop hook
- terminationGracePeriod for WebSocket drain

Dragonfly database (core presence)

1 primary + 2 replicas across AZs
AOF enabled
Automated failover via operator/orchestrator
Client connections:
- Service/virtual IP for reconnection
SLA characteristics:
- RPO bounded by async replication
- RTO at seconds-level

Dragonfly messaging (Pub/Sub + Streams)

Separate cluster, same topology (1 primary + 2 replicas, AOF)
Pub/Sub:
- Consumers auto-reconnect and resubscribe on fail-over
Streams:
- Consumers use consumer groups to resume from Pending Entry List

IAM (Keycloak)

2+ replicas behind the same L4 load balancer
OIDC endpoints highly available
Database:
- External HA (e.g., PostgreSQL with 1 primary + replicas, or managed)
Key operations:
- JWKS and token introspection served via LB
- API Gateway validates through LB to avoid single-node dependency

Observability

Prometheus pair scraping in parallel
Grafana:
- 2 replicas
- Persistent storage
Loki/Tempo:
- 2+ replicas for ingesters and queriers (or managed)
Alertmanager:
- HA mesh setup
Dashboards & alerts:
- Cover replica health, replication lag, failover KPIs

Operational HA practices:

Health Checks & Timeouts: strict readiness/liveness; conservative idle timeouts on LB to sustain long-lived WSS.
Rolling/Canary: phased rollouts for gateway and presence pods; error budgets gating production.
Backups & DR: periodic AOF snapshots exported to object storage; documented restore runbooks and drills.
Configuration: infra as code for quick re-provisioning; secrets stored in KMS/secret manager with replication.

Security

Communication with the client application is encrypted using WSS protocol and HTTPS (authentication flow).

In order to open a WebSocket connection and be able to access the system functionality, authentication is required with Keycloak. For increased security JWT is validated against the Keycloak token introspection endpoint.

API Manager provides rate-limiting and restricts message max size.

To protect Dragonfly unauthorized access the following measures are defined:

Authentication: the access is protected with authentication, with credentials configured for Online presence microservices to access Dragonfly.
Network security, like firewall or other kind of network-controls, to restrict network access to Dragonfly.
Data is encrypted in transit with TLS.

For preventing DDoS and other malicious attacks a Web Application Firewall can be used as first line of defense.

Submissions

Displaying Solutions with Scores Ranging from 1 to 10