Design a Serverless Architecture Framework

A serverless framework looks like simple function hosting, upload code, set a trigger, forget about servers. But behind that simplicity hide three real design challenges. Cold-start latency: spinning up an isolated container with runtime, code, and dependencies takes 100ms to 10s. Event routing at scale: normalizing HTTP requests, Kafka messages, S3 events, and cron triggers into a uniform invocation model. Scale-to-zero economics: containers must be reclaimed when idle (cost) but not so aggressively that every invocation pays a startup penalty (latency).

Functional

Deploy and invoke functions: Developers upload code bundles with a runtime specification (Node.js, Python, Java, Go) and the framework makes that function invocable within seconds. Each deployment produces an immutable version, enabling instant rollback if a new version fails.

10K RPS with 0.5s average execution means 5,000 concurrent containers consuming 1.28TB of memory. At peak, 1% cold starts mean 100 containers spinning up per second, each competing for CPU and network bandwidth to pull images and initialize runtimes.

Traffic

Average throughput: 10,000 invocations per second, translating to 864 million invocations per day. Peak throughput reaches 20K RPS during business hours, with burst capacity to 50K RPS for flash traffic events.

Two API surfaces serve fundamentally different consumers. The control plane (deploy, configure, monitor) handles low-traffic operations that require strong consistency. A deployment must be confirmed before the function is invocable. The data plane (invoke, trigger) handles high-traffic operations where latency matters more than acknowledgment. Conflating these two surfaces is the most common architectural mistake.

Why separate them? The control plane handles 100 requests per minute with authentication, validation, and database writes. The data plane handles 10,000 requests per second with minimal overhead. Routing both through the same gateway means deployment authentication middleware runs on every invocation. Adding latency to the hot path for security checks that only matter on the cold path.

Control Plane APIs

Four data types with four distinct access patterns: metadata (tiny, read on every invocation), code artifacts (large, read only on cold starts), logs (append-only, write-heavy), and invocation state (ephemeral, millisecond-lifetime). Each belongs in a specialized store, forcing them into one database means every store is wrong for three of the four patterns.

PostgreSQL: Function Metadata

1functions
2├── function_id UUID PRIMARY KEY
3├── tenant_id UUID NOT NULL (indexed)
4├── name VARCHAR NOT NULL
5├── runtime VARCHAR NOT NULL (nodejs18, python3.11, etc.)
6├── memory_mb INT NOT NULL (128-2048)
7├── timeout_sec INT NOT NULL (1-900)
8├── code_ref VARCHAR NOT NULL (S3 URI)
9├── version INT NOT NULL
10├── created_at TIMESTAMP NOT NULL
11└── updated_at TIMESTAMP NOT NULL
12
13triggers
14├── trigger_id UUID PRIMARY KEY
15├── function_id UUID FOREIGN KEY
16├── type VARCHAR NOT NULL (http, kafka, s3, cron)
17├── config JSONB NOT NULL (trigger-specific configuration)
18└── created_at TIMESTAMP NOT NULL

Six components, one separation principle. The control plane handles deployment (infrequent, latency-tolerant, strongly consistent. The data plane handles invocation) thousands per second, millisecond decisions, eventually consistent for metrics. Keeping them independent means a deployment storm never competes with invocations for resources.

Control Plane

Two flows with radically different profiles. Synchronous HTTP invocation is latency-critical (a user is waiting for the response. Asynchronous event processing is reliability-critical) the event must never be lost, even if processing is delayed. The architecture needs both a fast path (direct dispatch, minimal overhead) and a reliable path (Kafka queue, at-least-once delivery).

Synchronous Invocation Flow

How do you start a container, load a runtime, fetch code, initialize dependencies, and execute. All in under 500ms? At 10K RPS with 1% cold starts, 100 containers spin up per second. The answer is the warm container pool: pre-provisioning initialized runtimes so cold starts skip the most expensive steps.

GATE: Warm Container Pool with Predictive Pre-Warming

Three decisions, same underlying tension: cost versus latency. Containers versus microVMs, warm pools versus pure scale-to-zero, Kafka versus direct RPC. Each choice optimizes one dimension at the expense of the other. An interviewer asking about these trade-offs wants to see that you understand there is no universally correct answer. The right choice depends on the specific workload, security requirements, and cost constraints.

Containers versus Firecracker MicroVMs

Serverless failures are uniquely dangerous because users have no infrastructure visibility. They cannot SSH into a server, check logs locally, or restart a process. Every failure must be detected, mitigated, and communicated automatically. The platform must surface failures through structured logs, metrics dashboards, and alerting, giving developers the observability they lost by giving up server access.

Cold-Start Storm

A 10x traffic spike arrives. Hundreds of containers need provisioning simultaneously. All compete for CPU (runtime initialization), network (image pulls), and host memory (allocation). Instead of 500ms cold starts, each takes 10 seconds because resources are contention-bound.

The system handles 10K RPS with managed cold starts. These improvements push latency lower, costs down, and capabilities wider.

Edge Execution with V8 Isolates