DNS problem on AWS EKS when running in private subnets
Master System Design with Codemia
Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.
Understanding DNS Issues in AWS EKS with Private Subnets
Amazon Elastic Kubernetes Service (EKS) is a managed container service that allows you to run and scale Kubernetes applications in the cloud. When deploying EKS in AWS, particularly in a private subnet environment, you may encounter DNS resolution problems. This article delves into the potential DNS issues experienced when running EKS in private subnets, presents technical explanations, examples, and offers solutions.
Introduction to AWS EKS and Private Subnets
AWS EKS abstracts the Kubernetes control plane setup and management, allowing users to focus on running their containers. When deploying an EKS Cluster into a Virtual Private Cloud (VPC), resources can be launched in public, private, or hybrid subnets. Private subnets are commonly used for security reasons, but they have their unique challenges, especially related to network connectivity and DNS resolution.
DNS Resolution in AWS EKS
In Kubernetes clusters, CoreDNS is the primary DNS server, handling domain name resolution for service discovery. When running EKS in private subnets, DNS resolution must be appropriately configured to ensure that pods can reach the necessary endpoints, including external APIs, S3, ECR, and other AWS services.
Common DNS Issues
- CoreDNS FQDN Configuration:
- The default CoreDNS setup in EKS may not resolve Fully Qualified Domain Names (FQDN) correctly, especially for services hosted outside the cluster.
- DNS Traffic Not Routed:
- Traffic from nodes in private subnets to Amazon-provided DNS servers might not be routed because of misconfigured route tables or missing NAT Gateway configurations.
- VPC DHCP Option Set Misconfiguration:
- Incorrect settings in the VPC's DHCP option set can lead to DNS resolution failures, where nodes and pods cannot resolve needed external URLs.
- Network ACLs:
- Network Access Control Lists (ACLs) might block UDP packets, which are primarily used for DNS queries.
Technical Insights and Solutions
Verifying CoreDNS Configuration
Check the CoreDNS configuration in your EKS cluster using:
