How do I work out why an ECS health-check is failing?
System Design practice on Codemia
Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.
Introduction
An ECS health-check failure is not one thing. It can come from the container health check, the load balancer target group health check, or the task failing so early that neither check ever stabilizes. The fastest way to debug it is to identify which layer is declaring the task unhealthy, then inspect that layer directly.
Step 1: Identify Which Health Check Is Failing
With ECS services, two checks are common:
- a container health check defined in the task definition
- a target group health check from an Application Load Balancer or Network Load Balancer
These failures look similar in the console, but they mean different things. Start with ECS service events and task details.
Look for messages about unhealthy targets, task replacement, failed container checks, or tasks never reaching a steady state.
If the service uses a load balancer, inspect target health too:
That command often gives a clearer reason, such as wrong path, timeout, or connection failure.
Step 2: Check the Application From Inside the Task
If ECS or the load balancer says the health endpoint is failing, verify what the application actually returns.
If ECS Exec is enabled, connect to a running task and test locally:
You want to confirm three things:
- the application is listening on the expected port
- the health path is correct
- the endpoint returns a success status quickly enough
A surprising number of failures come from simple mismatches such as using / in the target group while the app exposes /health, or listening on 3000 while the service expects 8080.
Step 3: Inspect Logs and Exit Behavior
CloudWatch logs are often more useful than the health-check status itself. If the app crashes during startup, a failing health check is just a symptom.
Check for:
- binding failures on the expected port
- database connection timeouts during startup
- migrations that take too long
- unhandled exceptions before the HTTP server is ready
For container health checks, also inspect the exact command in the task definition. A shell-based check can fail because the binary is missing, the shell syntax is wrong, or the command returns a nonzero exit code even when the app is fine.
A typical container health check looks like this:
If curl is not present in the image, this health check fails every time regardless of application state.
Step 4: Validate Networking and Load Balancer Wiring
If the app is healthy locally but the target group still fails, the problem is often network or routing configuration.
Check these items carefully:
- security groups allow the load balancer to reach the task port
- the target group points at the correct container port
- the service is registered to the correct target group
- the health check path and protocol match the application
- the task is listening on all interfaces, not only on
127.0.0.1
That last point matters in containers. An app that listens only on loopback can respond to an in-container curl but still fail load balancer checks.
Step 5: Account for Startup Time
Some services are slow to become ready because they warm caches, run migrations, or connect to downstream systems. In those cases, the health-check parameters are too aggressive.
Look at:
- '
startPeriodfor container health checks' - target group health-check timeout and interval
- ECS service
healthCheckGracePeriodSeconds
If the app needs sixty seconds to become ready but the load balancer starts probing immediately, ECS may churn tasks forever even though the application would have stabilized with a grace period.
Common Pitfalls
The most common mistake is treating all ECS health-check failures as load balancer failures. Container and load balancer health are separate systems.
Another mistake is testing only from inside the container. That confirms app behavior, but not whether the load balancer can actually reach the task.
A third pitfall is using a health endpoint that depends on slow or fragile downstream services. Health checks should answer the readiness question quickly and predictably.
Summary
- First determine whether the failure comes from the container health check or the load balancer target group.
- Use
describe-services,describe-target-health, and application logs together. - Test the endpoint locally inside the task and verify the port, path, and status code.
- Validate security groups, target group wiring, and bind address configuration.
- If the app is slow to start, adjust grace periods and health-check timing rather than letting ECS churn tasks endlessly.
Related reading
- How do you attach and detach from Docker's process?
- How do you cleanly list all the containers in a kubernetes pod?
- How do you include postgresql.conf on docker container when using org.testcontainers
- How do you list volumes in docker containers?
- How do we query on a secondary index of dynamodb using boto3?
- How do we sort CloudWatch stream logs by 'most recent' in AWS console?
- How do you address messages coming out of order in a message queue?
- How do you catch this exception?

System Design Fundamentals
Build a strong foundation in designing scalable, reliable distributed systems.
View the courseTrack what you have practised
A free account saves your progress, solutions and study plan across every problem on Codemia.
System Design practice on Codemia
Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.