Docker BuildKit is an advanced build engine for Docker that offers improved performance, new features, and a more efficient way to build Docker images. However, when deploying applications in a managed Kubernetes service like AWS Elastic Kubernetes Service (EKS), users may encounter compatibility issues, particularly related to enabling Docker BuildKit. One common challenge is when the Docker daemon onboard does not support BuildKit. This article covers the technical reasons for this limitation, solutions, and best practices for working around these constraints in an AWS EKS Kubernetes environment.

Technical Overview

Docker BuildKit and Its Benefits

Docker BuildKit is an enhanced build tool that provides several benefits over the traditional Docker build process:

• Parallel Builds: BuildKit can build and cache multiple layers in parallel.
• Cache Import/Export: It supports advanced layer caching mechanisms.
• Smaller Images: By using multi-stage builds, it produces smaller, production-ready images.
• Advanced Syntax: With features like directives and secrets management directly in the Dockerfile.

Kubernetes and Container Runtimes

AWS EKS, like many Kubernetes platforms, typically defaults to using container runtimes compatible with Docker, but the actual runtime may vary. These can include:

• docker: The traditional Docker Engine.
• containerd: An industry-standard core container runtime that can be used independently of Docker.
• cri-o: Designed specifically for Kubernetes as part of the Open Container Initiative (OCI).

Because Docker BuildKit integrates more tightly with the Docker Engine, it may not be straightforward to use it directly when your Kubernetes cluster relies on alternative container runtimes or when BuildKit is not natively supported by the daemon.

Challenges in AWS EKS

Daemon Compatibility

The main issue arises from the fact that the Docker build command leverages a specific Docker API. If your AWS EKS cluster isn't using the Docker runtime directly or uses an older version, the Docker daemon may not support BuildKit.

In AWS EKS clusters configured with `containerd` or `cri-o` (the newer, standard defaults), the direct Docker features including BuildKit won't be natively available. This means developers might encounter errors or degraded build performance that prevents them from leveraging the full capabilities of BuildKit.

Current Limitations and Workarounds

1. Daemon Configuration: Most daemon configurations in AWS EKS aim towards security and resource efficiency, often excluding Docker in favor of less resource-hungry runtimes.
2. Buildkit Unavailability: Limited access to Docker's native build API means that BuildKit often remains unavailable or limited in its functionalities.

To work around this issue, some potential strategies include:

• Enable BuildKit in Docker Daemon: Ensure that your Docker installations in development and testing environments have the BuildKit feature enabled via `DOCKER_BUILDKIT=1` environment variable.
• Use Kaniko: Kaniko is an open-source tool designed for building container images in Kubernetes without requiring a privileged Docker daemon. Kaniko can perform similar caching and layering operations.
• Use a Dedicated Build Server: Consider performing Docker builds with BuildKit on a dedicated CI/CD server configured with Docker Engine that supports and enables BuildKit.

Example Implementation in EKS with Kaniko

Using Kaniko as an alternative to Docker builds directly in an AWS EKS cluster can alleviate some issues caused by the lack of BuildKit:

• name: kaniko
  • name: kaniko-secret
• name: kaniko-secret