Introduction

Hashing a file in Python means computing a deterministic digest from its bytes so you can compare files, verify integrity, or identify exact content without reading the whole file into memory at once. The usual tool is the standard library module hashlib, and the practical choice is almost always to stream the file in chunks.

Why Hash the File in Chunks

Small files can be read all at once, but large files should be hashed incrementally. That keeps memory usage stable and works the same way for a 1 MB file and a 10 GB file.

1import hashlib
2
3def sha256_file(path: str) -> str:
4    h = hashlib.sha256()
5    with open(path, "rb") as f:
6        for chunk in iter(lambda: f.read(8192), b""):
7            h.update(chunk)
8    return h.hexdigest()
9
10print(sha256_file("example.bin"))

This is the standard pattern because the hash object is designed to be updated incrementally.

Choosing the Hash Algorithm

Python supports several digest algorithms through hashlib, including MD5, SHA-1, SHA-256, and SHA-512.

For integrity checks in new code, SHA-256 is usually a sensible default:

import hashlib

print(hashlib.sha256(b"hello").hexdigest())

MD5 and SHA-1 are still common for non-security tasks such as duplicate detection or compatibility with legacy systems, but they should not be your first choice for security-sensitive uses.

Comparing a File Against a Known Digest

A common use case is checking whether a downloaded file matches an expected hash:

1expected = "9f86d081884c7d659a2feaa0c55ad015..."
2actual = sha256_file("download.zip")
3
4if actual == expected:
5    print("File is valid")
6else:
7    print("Hash mismatch")

This works because the hash changes drastically even if only one byte in the file changes.

Binary Mode Is Required

Always open the file in binary mode with "rb". Hash functions operate on bytes, not text-decoded characters.

If you open a file in text mode, line-ending translation or encoding behavior can change what is read, which means you may hash something different from the actual file bytes on disk.

Hashing Multiple Files Consistently

If you are hashing many files, keep the algorithm and output format consistent across the whole workflow. A mismatch between SHA-256 and MD5 is obvious, but smaller inconsistencies such as uppercase versus lowercase hex output or hidden whitespace in stored digests also cause confusion.

When performance matters, batch the filesystem traversal separately from the hashing logic so the hash function stays simple and testable.

Comparing Two Files Efficiently

If the goal is to check whether two files are identical, hashing both files and comparing digests is often simpler than writing a custom byte-by-byte comparison loop. It is still good practice to use the same algorithm and chunk size for both files so the comparison path stays predictable.

Wrap the Logic in a Reusable Helper

In real projects, put the hashing loop in one helper function and reuse it everywhere. That avoids subtle differences in chunk size, algorithm choice, or file-opening mode across the codebase.

Common Pitfalls

• Reading the whole file into memory when chunked hashing would be safer and simpler.
• Opening the file in text mode instead of binary mode.
• Using MD5 or SHA-1 in places where collision resistance matters.
• Comparing digests from different algorithms as if they were interchangeable.
• Treating hashing as encryption even though hashing is one-way and not reversible.

Summary

• Use hashlib to hash files in Python.
• Read files in binary mode and update the hash in chunks.
• SHA-256 is a strong general-purpose default.
• File hashing is useful for integrity checking, deduplication, and exact-content comparison.
• Keep the algorithm and digest format consistent across your workflow.