Amazon S3
Glacier
File Transfer
API
Cloud Storage

Move files between amazon S3 to Glacier and vice versa programmatically using API

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Introduction

With Amazon S3, "moving a file to Glacier" usually means changing the object's storage class, not copying it into a completely different product namespace. "Moving it back" is also easy to misunderstand: for archival classes such as S3 Glacier Flexible Retrieval and S3 Glacier Deep Archive, you normally restore the object first, and only then copy it into a warmer storage class if you want the change to be permanent.

That difference is the core of the API design. Archival is a storage-class transition, while retrieval is a restore workflow with latency, cost, and expiration rules.

Understand the Storage Classes First

S3 now has several archival-related classes, and they behave differently:

  • 'GLACIER_IR for S3 Glacier Instant Retrieval, which is immediately accessible'
  • 'GLACIER for S3 Glacier Flexible Retrieval, which requires restore before reading'
  • 'DEEP_ARCHIVE for the coldest and slowest retrieval path'

If your object is in GLACIER_IR, you do not "restore" it to read it. If it is in Flexible Retrieval or Deep Archive, you submit a restore request and wait for AWS to stage a temporary readable copy.

Archive an Object Programmatically

You can change an object's storage class by copying it over itself with a different StorageClass value.

python
1import boto3
2
3s3 = boto3.client("s3")
4bucket = "example-bucket"
5key = "reports/2024-01.csv"
6
7s3.copy_object(
8    Bucket=bucket,
9    Key=key,
10    CopySource={"Bucket": bucket, "Key": key},
11    StorageClass="GLACIER",
12    MetadataDirective="COPY",
13)
14
15head = s3.head_object(Bucket=bucket, Key=key)
16print(head.get("StorageClass"))

MetadataDirective="COPY" is important. Without it, you can accidentally replace metadata during the copy.

For large-scale archival, lifecycle policies are usually better than ad hoc API calls. The API approach is appropriate when a program decides storage class on demand.

Restore an Archived Object for Reading

For Flexible Retrieval or Deep Archive objects, restore is a separate operation. The restore does not permanently change the storage class. It creates a temporary restored copy for a chosen number of days.

python
1import boto3
2import time
3
4s3 = boto3.client("s3")
5bucket = "example-bucket"
6key = "reports/2024-01.csv"
7
8s3.restore_object(
9    Bucket=bucket,
10    Key=key,
11    RestoreRequest={
12        "Days": 3,
13        "GlacierJobParameters": {"Tier": "Standard"},
14    },
15)
16
17while True:
18    head = s3.head_object(Bucket=bucket, Key=key)
19    restore = head.get("Restore", "")
20    print(restore)
21    if 'ongoing-request="false"' in restore:
22        break
23    time.sleep(30)

Only after the restore completes should you try to download the object.

Permanently Move It Back to Standard Storage

A restore alone does not permanently reclassify the object. If you want the object to live in STANDARD again, restore it first and then copy it over itself with StorageClass="STANDARD".

python
1s3.copy_object(
2    Bucket=bucket,
3    Key=key,
4    CopySource={"Bucket": bucket, "Key": key},
5    StorageClass="STANDARD",
6    MetadataDirective="COPY",
7)

That second copy is the real "move back" step. Without it, the object will eventually leave the temporary restored state and remain in the original archive class.

Retrieval Tier and Cost Matter

Restore is not just a boolean operation. You choose a retrieval tier such as Expedited, Standard, or Bulk, depending on the archive class and the speed you need. Faster retrieval typically costs more.

This means programmatic restore logic should not be a blind one-liner. It should reflect the business urgency of the file being requested.

Know When Lifecycle Policies Are Better

If your policy is predictable, such as "archive logs after 30 days," use S3 lifecycle rules instead of custom code. Lifecycle policies are easier to audit, easier to reason about, and less likely to drift.

Use the API when the archive decision depends on application behavior, custom retention rules, or user-driven restore requests.

Common Pitfalls

The biggest mistake is treating S3 Glacier as if it were a separate folder that you can instantly move objects in and out of. Another is assuming restore_object permanently changes the storage class; it does not. Developers also often forget that restore for archive classes is asynchronous, so immediate download attempts fail. Finally, copy-based storage-class changes can unintentionally drop metadata if MetadataDirective is not handled correctly.

Summary

  • Archiving in S3 is usually a storage-class change, not a physical move to another service.
  • Use copy_object with a new StorageClass to archive or permanently reclassify an object.
  • For Flexible Retrieval and Deep Archive, restore_object creates temporary access only.
  • To move an archived object back permanently, restore it and then copy it to a warm class.
  • Lifecycle policies are usually better for predictable archival rules.

Course illustration
Course illustration

All Rights Reserved.