EC2
CloudWatch
STDOUT logs
AWS
logging

EC2 Instance - Sending STDOUT logs to Cloud Watch

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design

Introduction

EC2 does not automatically stream every process's stdout to CloudWatch Logs. The usual pattern is to make sure your application writes its stdout somewhere observable, such as a file or the system journal, and then configure the CloudWatch agent to collect that log source and send it to CloudWatch.

First decide where stdout actually goes

What "stdout logs" means depends on how the process is started:

  • a systemd service may send stdout to the journal
  • a shell script may redirect stdout to a file
  • a Docker container may keep stdout in the container runtime log stream

CloudWatch does not read from a vague concept of stdout. It reads from a real source you configure.

A common file-based setup

If your app is launched by a shell or supervisor, redirect stdout to a log file:

bash
python app.py >> /var/log/myapp/stdout.log 2>&1

Then configure the CloudWatch agent to tail that file.

Example agent config:

json
1{
2  "logs": {
3    "logs_collected": {
4      "files": {
5        "collect_list": [
6          {
7            "file_path": "/var/log/myapp/stdout.log",
8            "log_group_name": "/ec2/myapp/stdout",
9            "log_stream_name": "{instance_id}"
10          }
11        ]
12      }
13    }
14  }
15}

Once the agent is started with this config, new lines from the file are shipped to CloudWatch Logs.

IAM permissions are required

The EC2 instance needs an IAM role that lets the agent create log groups or streams and push log events. Without that, the agent can run but fail to publish anything useful.

At minimum, the role typically needs CloudWatch Logs actions such as:

  • 'logs:CreateLogGroup'
  • 'logs:CreateLogStream'
  • 'logs:PutLogEvents'

No amount of agent tuning fixes missing IAM permissions.

If the app runs under systemd

For Linux services, stdout may already be going to the journal. In that case, you can either collect journal logs with the CloudWatch agent or configure the service to append to a file that the agent tails.

The important part is consistency. Pick one path and make it obvious.

If the app is a Docker container on EC2

Be careful not to confuse EC2 instance logging with ECS or a Docker logging driver setup. If you are just running Docker directly on EC2, container stdout often ends up in Docker-managed logs on the host. You can collect those logs too, but the path and lifecycle differ from a plain host process.

That is why "send stdout to CloudWatch" is really a logging pipeline question, not just an EC2 checkbox.

Verify the pipeline end to end

After configuration:

  • confirm the app is writing to the expected source
  • confirm the CloudWatch agent is running
  • confirm the log group and stream appear
  • confirm new lines are arriving

Troubleshooting is much easier if you test with a known log line rather than waiting for natural app traffic.

Common Pitfalls

  • Assuming EC2 sends stdout to CloudWatch automatically with no agent or logging setup.
  • Forgetting that stdout has to land in a file, journal, or other concrete source first.
  • Missing IAM permissions for the CloudWatch agent.
  • Confusing Docker, EC2, ECS, and systemd logging paths.
  • Looking only at the application and forgetting to verify the agent and CloudWatch side too.

Summary

  • EC2 does not automatically ship arbitrary stdout to CloudWatch Logs.
  • Usually the app writes stdout to a file or journal, and the CloudWatch agent collects it.
  • The EC2 instance needs the right IAM role for log publishing.
  • The logging path depends on whether the app runs directly, under systemd, or inside Docker.
  • Think of this as a full logging pipeline: source, collector, permissions, and destination.

Related reading
Course
Beginner
27 lessons
10 hours
System Design Fundamentals

Build a strong foundation in designing scalable, reliable distributed systems.

View the course
Track what you have practised

A free account saves your progress, solutions and study plan across every problem on Codemia.

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design

All Rights Reserved.