Introduction

You cannot make an AWS Spot instance truly persistent in the same way as an On-Demand instance. Spot capacity is interruptible by design, so the practical goal is not to prevent interruption entirely, but to make the workload, storage, and recovery path persistent enough that interruption is cheap.

Start with the Right Mental Model

A Spot instance is disposable compute. If the application depends on the machine itself being immortal, Spot is the wrong abstraction.

The useful question is:

"How do I keep progress, data, and service availability even when the instance disappears?"

That shifts the design toward resilient infrastructure rather than trying to defeat the Spot model.

Keep Data Off the Instance

The first rule is to separate state from compute. Important data should live in durable services such as:

• EBS volumes
• S3
• RDS
• DynamoDB
• EFS

For example, user uploads or model checkpoints should not live only on the instance store. If the node is reclaimed, that local data is gone.

A common pattern is to write checkpoints to S3 periodically:

aws s3 cp checkpoint.bin s3://my-bucket/checkpoints/checkpoint.bin

That way a replacement instance can resume work instead of starting over.

Use Auto Scaling or a Fleet

To keep capacity around even when individual Spot instances disappear, run them in an Auto Scaling group or another fleet-style controller. AWS can then replace interrupted instances automatically.

The persistence comes from the group maintaining desired capacity, not from a single instance surviving forever.

This is especially effective when you diversify across:

• multiple instance types
• multiple Availability Zones
• mixed Spot and On-Demand capacity

Diversification reduces the chance that one capacity shortage wipes out the whole workload at once.

Handle the Interruption Notice

Spot interruptions usually provide a short warning window. Your software should treat that notice as a trigger to flush state, stop accepting new work, or deregister from load balancers.

A simple polling example from inside the instance is:

curl -s http://169.254.169.254/latest/meta-data/spot/instance-action

If the endpoint returns an interruption action, your shutdown script can checkpoint work and exit cleanly.

Design the Workload to Resume

Persistent behavior usually comes from resumability. Batch jobs, render pipelines, CI runners, and data-processing workers should save progress frequently and process work in chunks small enough to retry.

For example:

• store queue state in SQS
• checkpoint training progress regularly
• keep job ownership in a database rather than in process memory

If the instance dies, another instance picks up where it left off.

Consider Hibernation Only for Narrow Cases

Some Spot workloads can use hibernation, which preserves memory to the root EBS volume and resumes later if conditions allow. This can help certain long-running processes, but it is not a general persistence guarantee and has specific eligibility requirements.

It is best viewed as a convenience optimization, not as a substitute for durable architecture.

Mix Spot with On-Demand When Availability Matters

If the workload must stay alive continuously, mix in some On-Demand capacity. A common pattern is:

• baseline service on On-Demand
• burst or background capacity on Spot

That gives you savings without making the whole system fragile.

Common Pitfalls

The biggest mistake is trying to make one individual Spot instance permanent. AWS can reclaim it, and no client-side trick changes that contract.

Another mistake is storing important state only on the local disk of the Spot instance. That defeats the whole recovery story.

Developers also forget to test interruption handling. A checkpoint system that works in theory but fails during a real termination is not enough.

Finally, do not run a business-critical singleton service only on Spot unless you are comfortable with interruption risk. Persistence at the system level often requires mixed capacity or redundancy.

Summary

• A Spot instance itself is not persistent; design for replacement instead.
• Store important state in durable services such as S3, EBS, EFS, or databases.
• Use Auto Scaling groups or fleets so interrupted instances are replaced automatically.
• Handle interruption notices by checkpointing and shutting down cleanly.
• For higher availability, combine Spot with some On-Demand capacity.