System requirements

Functional:

• User Authentication:
• Allow users to securely authenticate using credentials, MFA, or SSO.
• Support various authentication methods such as password-based, biometric, or token-based authentication.
• Access Control:
• Enforce role-based access control to manage user permissions and privileges.
• Ensure secure resource access based on user roles and policies.
• Identity Federation:
• Enable users to seamlessly federate their identities across multiple services or organizations.
• Support federation standards like SAML (Security Assertion Markup Language) or OAuth.
• User Lifecycle Management:
• Manage user accounts throughout their lifecycle, including creation, modification, and deactivation.
• Implement processes for user onboarding, offboarding, and account recovery.
• Audit Logging:
• Log all identity-related events, including authentication attempts, access grants, and policy changes.
• Maintain audit trails for compliance and security analysis purposes.

Non-Functional:

• Security:
• Ensure data encryption in transit and at rest to protect sensitive user information.
• Implement secure communication protocols such as HTTPS for data transmission.
• Maintain compliance with industry standards like GDPR, HIPAA, or PCI DSS.
• Scalability:
• Design the system to scale horizontally to accommodate growing numbers of users and resources.
• Ensure high availability and performance under increasing user loads.
• Usability:
• Provide a user-friendly interface for user authentication and account management.
• Support accessibility standards to cater to users with diverse needs.
• Reliability:
• Minimize system downtime and ensure fault tolerance in case of failures.
• Implement backup and recovery strategies to protect user data and system configurations.
• Performance:
• Optimize system response times for authentication and access control operations.
• Conduct performance testing to validate system efficiency under varying loads.
• Isolation:
• Isolation is one of the most important non-functional requirements for this service.
• Most client would not tolerate sharing the same database or even the same service with another client.

Capacity estimation

• Number of Users: millions of users
• Authentication Requests: 10000 authentication requests per second
• Read Operations: 5000 read operations per second for user data retrieval
• Write Operations: 500 write operations per second for user updates
• Storage Requirements:
• User Profile: 10 MB per user
• Authentication Data: 1000 KB per authentication record
• Audit Logs: 10 KB per log entry
• Network Bandwidth:
• Expected network traffic: 10 Gbps
• Real-time communication: Use 100 Mbps for WebSocket connections

API design

Authentication Endpoints:

• Endpoint for user authentication. Accepts user credentials and returns an access token upon successful authentication.

• Endpoint for user logout. Invalidates the current session token.

• Endpoint to refresh the access token with a new token after its expiration.

User Management Endpoints:

• Retrieves a list of users from the user database.

• Creates a new user in the system.

• Retrieves details of a specific user by ID.

• Updates the user information for a specific user.

• Deletes a user from the system.

Access Control Endpoints:

• Retrieves a list of available roles in the system.

• Creates a new role.

• Retrieves a list of available permissions.

• Creates a new permission.

• Assigns a role to a user.

Federation Endpoints:

• Initiates a federation request for user identity verification.

• Verifies the identity of a federated user.

Audit Logging Endpoints:

• Retrieves audit logs related to user activities and system events.

• Records a specific event or log entry in the audit trail.

Real-time Communication Endpoints:

• Retrieves real-time notifications for the user.

• Sends a real-time update to the system or specific user.

• Sends a real-time alert to the system for immediate action.

Database design

• User Table:
• Store user information such as user ID, username, password (hashed), email, name, and other profile details.
• Include columns for user roles, permissions, and status (active/inactive).
• Role Table:
• Define roles that users can be assigned to, such as admin, manager, or regular user.
• Include role ID, role name, and description columns.
• Permission Table:
• List out permissions that define what actions a user with a specific role can perform.
• Include permission ID, permission name, and description columns.
• User_Role Table (Many-to-Many Relationship):
• Connect users to their assigned roles using a junction table.
• Include user ID and role ID columns to establish the relationship between users and roles.
• User_Permission Table (Many-to-Many Relationship):
• Associate user roles with the permissions they have.
• Connect roles to permissions through a junction table with role ID and permission ID columns.
• Audit Log Table:
• Capture audit logs for user activities, login attempts, access grants, and policy changes.
• Include columns for log ID, timestamp, user ID, action performed, and additional details.
• Session Table:
• Manage user sessions and store session tokens for authentication and authorization.
• Include columns for session ID, user ID, token, creation timestamp, and expiration timestamp.
• Federation Table:
• Handle federation information such as federated user IDs, external service IDs, and other relevant identity data.
• Include columns for federation ID, user ID, service provider, and federation status.

High-level design

Cell-Based Architecture

Okta uses cell based architecture to partition its services into self-contained units called "cells". Each cell operates independently, encapsulating specific functions and services. This isolation at the cell level helps contain failures and improves fault tolerance.

Kubernetes Orchestration

By leveraging Kubernetes for container orchestration, Okta can deploy, manage, and scale containerized services within each cell. Kubernetes provides features for service discovery, load balancing, health checks, and auto-scaling, enhancing the efficiency and reliability of the system.

Isolation within Cells

Within each cell, Kubernetes enforces container isolation using features like namespaces and resource quotas. This ensures that services within a cell are isolated from each other and have defined resource constraints for enhanced security.

Secure Multi-Tenancy

The cell-based architecture enables Okta to support secure multi-tenancy by segregating customer data and workloads into separate cells. Each cell operates independently, serving a specific set of users or customers while ensuring data and resource isolation.

Scalability and Resilience

Kubernetes' scalability features, such as horizontal pod autoscaling and self-healing capabilities, allow Okta to dynamically adjust resource allocation based on workload demands. This ensures high availability and optimal performance across cells.

Security and Compliance

Kubernetes provides a robust security model with built-in mechanisms for network policies, identity and access management, encryption at rest and in transit, and auditing capabilities. This helps Okta maintain security and compliance standards within each cell.

In a cell-based architecture utilizing Kubernetes, each cell typically encapsulates a set of services or components that work together to perform specific functions within the system. While the exact composition of services within each cell may vary based on the architecture and requirements of the identity management system, here is a generalized example of services that could be included within a cell:

In the case of Okta, every cell is an isolated, shared-nothing, identical replica of our infrastructure, spanning from the bottom layer all the way to our edge. Each cell is a self-contained instance of the entire Okta service

Below are a list of services that each cell would contain.

Authentication Service:

• Responsible for user authentication, credential verification, and token generation.

Authorization Service:

• Manages user permissions, access control policies, and authorization requests.

Identity Provider:

• Authenticates users through various methods such as passwords, biometrics, or tokens.
• Manages user sessions and issues access tokens upon successful authentication.
• Supports secure authentication protocols like OAuth, OpenID Connect, or SAML.
• Integrates with external identity providers for federated authentication.

Resource Server:

• Hosts resources (e.g., files, services, applications) that users want to access.
• Enforces access control policies based on user permissions and roles.
• Implements fine-grained access control mechanisms to protect sensitive resources.
• Logs access attempts and actions for auditing and compliance.

User Database:

• Stores user profiles, authentication credentials, roles, and permissions.
• Ensures data integrity, confidentiality, and availability.
• Encrypts sensitive user data at rest and in transit.
• Implements backup and recovery strategies to protect user information.

Single Sign-On Service:

• Enables users to authenticate once and access multiple services seamlessly.
• Synchronizes user sessions across applications for a unified user experience.
• Supports various SSO protocols like SAML, OAuth, or OpenID Connect.
• Integrates with authentication providers to streamline login experiences.

Multi-factor Authentication Service:

• Adds an extra layer of security by requiring users to provide multiple authentication factors.
• Verifies user identities using methods like SMS codes, biometrics, or authenticator apps.
• Implements adaptive authentication to assess risk and adjust security measures accordingly.
• Allows users to manage MFA settings and devices securely.

User Lifecycle Management Service:

• Manages user accounts throughout their lifecycle, from creation to deletion.
• Handles user profile updates, role assignments, and account suspensions.
• Automates onboarding and off-boarding processes for efficient user management.
• Generates reports on user activity, account status, and access permissions.

Role-Based Access Control Service:

• Defines roles, permissions, and access levels for users based on their organizational roles.
• Enforces access policies to ensure users have appropriate privileges for resource access.
• Supports hierarchical role structures and role assignment mechanisms.
• Implements least privilege access to minimize security risks.

Federation Service:

• Facilitates identity federation processes between the Identity Provider and Service Provider.
• Manages the exchange of authentication and identity information between disparate systems.
• Implements standards like Security Assertion Markup Language (SAML) or OAuth for secure federation.
• Supports trusted relationships with external identity providers for seamless user authentication.

WebSocket Connections:

Enable real-time communication and data exchange between components for efficient updates, notifications, and alerts.

Request flows

1. The user initiates a request to access a resource hosted on the Resource Server.
2. The user is redirected to the Identity Provider for authentication.
3. Once authenticated, a federation request is made to the Federation Service for access.
4. The Federation Service verifies the user's identity, retrieves user information from the User Database, and handles operations related to SSO, MFA, user lifecycle management, and role-based access control.
5. Finally, upon successful authorization and user verification, the Resource Server grants access to the requested resource.

Detailed component design

In order to implement our Cell-based architecture we can leverage existing solutions such as Cellery. Cellery leverages Kubernetes as the underlying container orchestration system to deploy and manage microservices-based applications. Cellery has a unique approach to building and orchestrating composite applications as connected sets of microservices. Cellery aims to simplify the development, deployment, and management of complex distributed systems by providing a higher-level abstraction that focuses on the composition and interactions of microservices. It abstracts away some of the complexity of directly working with Kubernetes manifests and resources, making it easier to work with microservices in a Kubernetes environment.

Cell

In Cellery, a 'Cell' is a unit of deployment that encapsulates one or more microservices along with the dependencies and configurations required to run them. Cells provide a way to package and deploy related microservices as a single entity, simplifying management and versioning.

Cell Image

A 'Cell Image' is a versioned artifact that contains the definition of a Cell, including the microservice components, dependencies, configurations, and metadata. Cell Images can be pushed to a Cellery repository for sharing and deployment.

Cell Gateway

The 'Cell Gateway' serves as the entry point for incoming requests to a Cellery-based application. It handles routing, load balancing, and security aspects, allowing external clients to interact with the composite application.

Observability

Cellery provides tools and capabilities for monitoring, tracing, and logging the interactions within composite applications. It enables developers and operators to gain insights into the behavior and performance of interconnected microservices

Benefits of Using Cellery:

• Simplified Composition: Cellery simplifies the process of composing and managing interconnected microservices within composite applications.
• Consistent Deployment: Cells ensure consistent deployment and versioning of related microservices as a single unit.
• Enhanced Observability: Built-in observability features help in monitoring the interactions and performance of composite applications.
• Security Enforcement: Cell Gateway facilitates secure communication and access control for external interactions with the composite application.

Implementation Steps using Cellery:

1. Define Composite Application:
2. Identify the microservices that need to be included in the composite application and define the interactions and dependencies between them.
3. Create Cell Definition:
4. Define a Cell using Cellery's Cell Model, specifying the microservices, dependencies, configurations, and other metadata required for the composite application.
5. Build Cell Image:
6. Build a versioned Cell Image using the defined Cell, which encapsulates all the components and configurations needed to run the composite application.
7. Push Cell Image:
8. Push the built Cell Image to a Cellery repository to make it available for deployment across different environments.
9. Compose Composite Application:
10. Compose a composite application by deploying multiple Cells together, ensuring that their interactions and dependencies are correctly configured.
11. Deploy Composite Application:
12. Deploy the composed composite application using Cellery runtime, which manages the deployment, scaling, and lifecycle of the interconnected microservices.
13. Configure Cell Gateway:
14. Configure the Cell Gateway to serve as the entry point for external requests to the composite application, managing routing, load balancing, and security aspects.
15. Monitor and Manage:
16. Utilize Cellery's built-in monitoring and observability features to monitor the interactions, performance, and health of the composite application.

When a user makes a request to create a cell in Cellery, the process involves defining the configuration for the composite application containing the microservices that make up the cell. This configuration typically includes the specifications for the individual microservices, their connections, dependencies, and any other required resources.

Let's break down the high-level steps involved in the process of creating a cell in Cellery:

1. Define the Cell Manifest: The user starts by creating a Cell Manifest, which describes the composition of the cell, including the microservices it consists of, their configurations, dependencies, and any other relevant details.
2. Build and Package Microservices: Each microservice within the cell needs to be built and packaged as a container image. This can involve compiling the code, adding necessary dependencies, and creating the Docker image for each microservice.
3. Compose the Cell: The cell configuration is used to compose the individual microservices into a single unit called a cell. This composition specifies how the microservices interact with each other and any other resources needed.
4. Deploy the Cell to Kubernetes: Once the cell is composed, it can be deployed to the Kubernetes cluster. Cellery interacts with Kubernetes to create the necessary resources (pods, services, etc.) to run the cell in the cluster.
5. Start and Manage the Cell: After deployment, Cellery ensures that the cell is running correctly in the Kubernetes environment. It monitors the health of the microservices, manages communication between them, and handles any scaling or updating requirements.

Individual components

We would design individual components/services for each cell as below:

Authentication Component:

We can break down the Authentication Service into further subcomponents like User Authentication Methods, Token Management, and Security Measures. Let's create a detailed diagram focusing on these aspects:

• User: Initiates the authentication process by providing login credentials.
• Authentication Service: Validates user credentials using various authentication methods such as username/password or multi-factor authentication.
• User Database: Stores user information and authentication data securely for verification.
• Multi-factor Authentication Service: Enhances security by requiring additional authentication factors.
• Token Management: Handles the generation and management of access tokens for secure resource access.
• Resource Server: Receives and validates access tokens to grant access to protected resources.
• User Database (Refresh Token): Stores and manages refresh tokens to issue new access tokens.
• Security Measures: Includes encryption of sensitive data, rate limiting to prevent brute force attacks, and audit logs for monitoring and compliance.

Authorization and Access Control:

We can break down the Authorization and Access Control into subcomponents such as Role-Based Access Control (RBAC), Policies Enforcement, and Secure Resource Sharing. Let's create a detailed diagram focusing on these aspects:

• User: Interacts with the system to access resources that are protected by authorization rules.
• Authorization Service: Responsible for checking permissions and enforcing access control policies.
• Access Control:
• Role-Based Access Control (RBAC): Assigns roles to users and grants permissions based on roles.
• Attribute-Based Access Control: Considers user attributes and environmental conditions for access decisions.
• Role Management: Manages roles assigned to users and their corresponding permissions.
• Policy Management: Defines and manages access control policies based on attributes and conditions.
• Policy Engine: Enforces policies and makes access control decisions.
• Logging Service: Maintains audit logs for tracking and monitoring access control activities.
• Resource Sharing Server: Facilitates secure sharing of resources based on access control rules.

Identity Federation:

Identity Federation plays a crucial role in enabling seamless and secure access to resources across multiple trusted organizations or systems.

• User: Initiate the authentication process to access resources seamlessly across different systems.
• Single Sign-On (SSO): Allows users to log in once and access multiple applications without re-entering credentials.
• Identity Provider: Manages user identities, authenticates users, and issues tokens for access.
• External Identity Provider: Represents identity providers from other trusted organizations participating in the federation.
• Trust Relationship: Establishes trust between the Identity Provider and External Identity Providers to enable secure authentication.

The process typically involves the following steps:

1. The user initiates the authentication process by accessing a resource.
2. The Identity Provider authenticates the user and issues an authentication token.
3. If the resource is hosted by an External Identity Provider, a trust relationship allows the user to be authenticated by that provider.
4. The user gains access to the requested resource without the need for repeated authentication.

Identity Federation simplifies user access management, enhances user experience, and ensures secure authentication across diverse systems.

Real-time Communication:

• Notification Service: Provides real-time notifications to users regarding authentication events, authorization approvals, or security alerts.
• Authentication Service: Validates user credentials, verifies user identity, and issues authentication tokens.
• User: Receives notifications and authentication responses in real-time.
• Resource Server: Enforces access control policies and grants access to authorized users.

Trade offs/Tech choices

Explain any trade offs you have made and why you made certain tech choices...

Failure scenarios/bottlenecks

Some potential bottlenecks include:

• High token generation rates may overload the Token Service, causing delays in authentication.
• Heavy policy evaluation processes may slow down access control decisions, impacting system responsiveness.

Future improvements