Design a Secure Identity Management System

A user clicks "Sign in with Google" and lands in your app authenticated, authorized, and scoped to exactly the resources their role permits, all in under 200ms. Behind that click: an OAuth 2.0 authorization code exchange, an MFA challenge, a JWT minted with role claims, and an RBAC policy evaluation. This system is the gatekeeper for every other service in the organization. If it goes down, every service goes dark.

Functional

User registration and profile management: Users create accounts with email and password, manage profile data, and link external identity providers. The system stores credentials securely using bcrypt with a cost factor of 12, which takes roughly 250ms per hash, slow enough to resist brute force, fast enough for interactive login.

The numbers for an identity system look deceptively small until you realize that token validation (not login) is the dominant operation. 50M users logging in 5 times per day is 250M login requests, roughly 2,900 per second at steady state and 15,000 at peak. But every API call across every service must validate a token. If 50M users make 50 API calls per day each, that is 2.5 billion token validations per day, roughly 29,000 per second. The validation path must be sub-millisecond, which means local JWT signature verification, not a round-trip to the IdP.

Traffic

Login requests: 50M users, 5 logins per day average = 250M daily. 250M / 86,400 seconds = 2,900 logins per second average. Peak at 5x = 14,500 per second. Each login involves a bcrypt hash (250ms), an MFA challenge (varies), and a JWT issuance (under 1ms). The bcrypt step is CPU-bound: at 2,900 logins per second, you need 725 CPU-seconds of bcrypt capacity per second, meaning at least 725 CPU cores dedicated to password hashing at steady state.

Six endpoints define the identity system, but the security decisions live in the details. POST /login does not return a token directly, it returns an MFA challenge. POST /mfa/verify completes authentication and issues the JWT. This two-step pattern ensures no token is ever issued without MFA verification, even if the password is compromised.

Authentication Endpoints

1POST /register
2Body: { email, password, name }
3Response: { user_id, mfa_setup_url }
4Notes: Password is bcrypt-hashed server-side (cost factor 12).
5       Returns MFA setup URL (TOTP QR code) for immediate enrollment.
6       Rate limited to 10 requests per IP per hour.
7
8POST /login
9Body: { email, password }
10Response: { mfa_challenge_id, mfa_type, expires_in }
11Notes: Does NOT return a token. Validates password, then returns
12       an MFA challenge. The challenge expires in 5 minutes.
13       Returns identical error for wrong email or wrong password
14       to prevent username enumeration.
15
16POST /mfa/verify
17Body: { mfa_challenge_id, code }
18Response: { access_token, refresh_token, expires_in }
19Notes: Validates TOTP code against stored seed. On success,
20       issues JWT access token (15 min TTL) and refresh token
21       (7 day TTL). Refresh token stored in Redis.
22
23POST /token/refresh
24Body: { refresh_token }
25Response: { access_token, refresh_token, expires_in }
26Notes: Validates refresh token against Redis. Issues new access
27       token AND new refresh token (rotation). Old refresh token
28       is invalidated immediately. If a rotated-out refresh token
29       is reused, all tokens for that user are revoked (theft
30       detection).
31
32POST /logout
33Headers: Authorization: Bearer <access_token>
34Body: { refresh_token }
35Response: { success: true }
36Notes: Adds access token to Redis revocation list (TTL = remaining
37       token lifetime). Deletes refresh token from Redis.

Two data categories with opposite access patterns define the storage architecture. User profiles and roles are read-heavy, consistency-critical, and small (500GB), PostgreSQL with read replicas. Audit logs are write-heavy, append-only, and massive (15TB per month), TimescaleDB with time-based partitioning and TTL.

Core Tables (PostgreSQL)

1users
2├── user_id        UUID PRIMARY KEY
3├── email          VARCHAR(255) UNIQUE NOT NULL
4├── password_hash  CHAR(60) NOT NULL  -- bcrypt output
5├── name           VARCHAR(255)
6├── status         ENUM('active','suspended','deactivated')
7├── created_at     TIMESTAMP
8└── updated_at     TIMESTAMP
9
10roles
11├── role_id        UUID PRIMARY KEY
12├── name           VARCHAR(100) UNIQUE  -- 'admin','editor','viewer'
13├── description    TEXT
14└── created_at     TIMESTAMP
15
16user_roles
17├── user_id        UUID REFERENCES users
18├── role_id        UUID REFERENCES roles
19├── granted_by     UUID REFERENCES users
20├── granted_at     TIMESTAMP
21└── PRIMARY KEY (user_id, role_id)
22
23role_permissions
24├── role_id        UUID REFERENCES roles
25├── resource       VARCHAR(255)  -- 'articles','users','settings'
26├── action         VARCHAR(50)   -- 'read','write','delete'
27└── PRIMARY KEY (role_id, resource, action)
28
29auth_factors
30├── factor_id      UUID PRIMARY KEY
31├── user_id        UUID REFERENCES users
32├── factor_type    ENUM('totp','webauthn','sms','backup_codes')
33├── secret         BYTEA  -- encrypted TOTP seed or WebAuthn public key
34├── is_primary     BOOLEAN DEFAULT false
35├── created_at     TIMESTAMP
36└── last_used_at   TIMESTAMP
37
38oauth_clients
39├── client_id      UUID PRIMARY KEY
40├── client_secret  CHAR(60)  -- bcrypt-hashed secret
41├── name           VARCHAR(255)
42├── redirect_uris  TEXT[]  -- allowed callback URLs
43├── allowed_scopes TEXT[]  -- 'openid','profile','email'
44├── is_active      BOOLEAN DEFAULT true
45└── created_at     TIMESTAMP

The architecture centers on a stateless Auth Service behind an API Gateway, with three storage layers optimized for their access patterns.

Components

Login and MFA Flow

The login flow is deliberately two-step: password validation first, then MFA verification. No token is ever issued after just a password check.

OAuth 2.0 and SAML Federation: The GATE

Federation is what transforms a login system into an identity platform. Without federation, every service maintains its own user database, its own login page, and its own password policy. With federation, a single Identity Provider (IdP) authenticates users and issues tokens that every service trusts. The GATE insight: OAuth 2.0/SAML federation collapses N identity silos into one, making the IdP's availability and security the foundation of the entire organization.

Token TTL: Short vs. Long

Short TTL (15 minutes): A stolen token is useful for at most 15 minutes. But users refresh frequently: 10M concurrent users refresh 4 times per hour = 40M refreshes per hour = 11,000 per second. This puts sustained load on Redis and the Auth Service.

IdP Outage

Impact: No new logins across any service. Existing sessions with valid JWTs continue working. Token refreshes fail, so users whose tokens expire during the outage lose access.

Mitigation: Active-passive IdP cluster with automatic failover. The standby instance shares the same PostgreSQL database (or a synchronous replica). Failover time target: under 30 seconds. During failover, in-flight login requests fail and clients retry. With 15-minute tokens, the fraction of sessions affected equals the outage duration divided by the token TTL. A 30-second outage affects roughly 1/30th of sessions (those whose tokens happen to expire during that window). A 5-minute outage affects one-third. This is why fast failover matters.

Passwordless Authentication (FIDO2/WebAuthn)

Passwords are the weakest link in any identity system, they can be phished, reused, and brute-forced. FIDO2/WebAuthn replaces passwords entirely with public-key cryptography bound to a hardware authenticator or platform biometric. The user registers a passkey (laptop fingerprint reader, phone face recognition, hardware security key), and the private key never leaves the device. Login becomes: server sends a challenge, device signs it, server verifies the signature. No shared secret, no phishing surface, no password database to breach. The migration path: offer passkey registration alongside password login, then gradually enforce passkey-only for new accounts.

Adaptive Risk-Based MFA