AWS EC2
job scheduling
cloud computing
Amazon Web Services
automation

Scheduling A Job on AWS EC2

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design

Introduction

If a task must run on an EC2 instance at a fixed time, the first decision is whether the schedule should live inside the instance or be managed by AWS outside the instance. For simple host-local jobs, cron or a systemd timer is usually enough; for centrally managed scheduling, AWS tools such as EventBridge and Systems Manager are often better.

The Simple EC2 Answer: Use cron

If the job only needs to run on that specific Linux instance, cron is still the most direct solution.

Create a shell script:

bash
1#!/usr/bin/env bash
2set -euo pipefail
3
4cd /opt/myapp
5/usr/bin/python3 backup.py >> /var/log/myapp-backup.log 2>&1

Make it executable:

bash
chmod +x /opt/myapp/run-backup.sh

Then edit the crontab:

bash
crontab -e

Run the job every day at 02:00:

cron
0 2 * * * /opt/myapp/run-backup.sh

That is often the right answer when the instance is long-lived and the job belongs to that machine.

Use Full Paths and Explicit Environment

Jobs that work in an interactive shell often fail in cron because the environment is smaller. Use full command paths and set required environment variables explicitly.

cron
1SHELL=/bin/bash
2PATH=/usr/local/bin:/usr/bin:/bin
3AWS_REGION=us-east-1
4
50 2 * * * /opt/myapp/run-backup.sh

That avoids a large class of "works over SSH, fails in cron" problems.

systemd Timers Are a Strong Alternative

On modern Linux distributions, systemd timers are often easier to observe and manage than cron.

Service unit:

ini
1[Unit]
2Description=Run nightly backup
3
4[Service]
5Type=oneshot
6WorkingDirectory=/opt/myapp
7ExecStart=/usr/bin/python3 /opt/myapp/backup.py

Timer unit:

ini
1[Unit]
2Description=Nightly backup timer
3
4[Timer]
5OnCalendar=*-*-* 02:00:00
6Persistent=true
7
8[Install]
9WantedBy=timers.target

Enable the timer:

bash
sudo systemctl enable --now myapp-backup.timer

This approach integrates nicely with journalctl and systemctl list-timers.

When AWS-Managed Scheduling Is Better

If you want the schedule defined in AWS rather than on the machine, use an external trigger. A common pattern is:

  • EventBridge defines the schedule
  • Systems Manager Run Command executes a script or command on the instance

That is better when:

  • the instance fleet changes over time
  • you want central visibility into schedules
  • you want to trigger jobs without SSH access

From an architecture perspective, this keeps orchestration in AWS and execution on the instance.

What Not to Do

A common misconception is that "schedule a job on EC2" automatically means "use Lambda." Lambda is great for many scheduled tasks, but it is the wrong tool if the task must run inside the EC2 instance because it depends on that instance's local files, processes, or mounted storage.

If the work is really just "run code on a schedule," consider whether the job should live in Lambda instead of EC2 at all. If the work is truly instance-bound, keep it on the instance or trigger it through Systems Manager.

Logging and Monitoring Matter

No scheduler is useful if failures disappear silently. At minimum:

  • redirect cron output to a log file or logging system
  • make scripts exit non-zero on failure
  • monitor logs or emit metrics
  • think about time zone expectations, especially if the instance runs in UTC

For example:

bash
/usr/bin/python3 backup.py >> /var/log/myapp-backup.log 2>&1

That tiny detail is the difference between easy diagnosis and blind debugging.

Common Pitfalls

The biggest pitfall is relying on relative paths inside scheduled scripts. Cron and service managers do not necessarily start in the directory you expect.

Another pitfall is forgetting that the instance time zone controls the schedule for cron and many timer-based setups. If the business requirement says "every day at 2 a.m. New York time," verify the instance clock and deployment region behavior carefully.

A third pitfall is scheduling important jobs on ephemeral or auto-scaled instances without thinking about lifecycle. If the instance is replaced, the schedule or local state may disappear unless the design accounts for that.

Finally, avoid manual SSH-only operations as your scheduling strategy. If the task matters, script it, log it, and make it observable.

Summary

  • Use cron for simple instance-local recurring jobs on EC2
  • Use systemd timers when you want better service management and observability
  • Use EventBridge plus Systems Manager when you want AWS-managed scheduling outside the instance
  • Be explicit about paths, environment variables, logging, and time zones
  • Choose the scheduler based on where the job logically belongs, not just on which AWS service sounds familiar

Related reading
Course
Beginner
27 lessons
10 hours
System Design Fundamentals

Build a strong foundation in designing scalable, reliable distributed systems.

View the course
Track what you have practised

A free account saves your progress, solutions and study plan across every problem on Codemia.

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design

All Rights Reserved.