Introduction

In Python 3, text strings are Unicode, but files on disk are bytes. Reading and writing UTF-8 correctly is therefore about making the text-to-bytes conversion explicit, so Python does not guess the wrong encoding from the operating system or runtime defaults.

Writing UTF-8 Text Explicitly

The safest pattern is to pass encoding="utf-8" whenever you open a text file.

1text = "Hello, café, こんにちは, Привет"
2
3with open("example.txt", "w", encoding="utf-8") as f:
4    f.write(text)

This tells Python exactly how to encode the Unicode string into bytes before writing to disk. It also makes the code more portable because the behavior does not depend on the platform's default text encoding.

Reading UTF-8 Text Back

Use the same explicit encoding when reading:

1with open("example.txt", "r", encoding="utf-8") as f:
2    content = f.read()
3
4print(content)

If the file really is UTF-8 encoded, this gives you back a normal Python str containing Unicode text.

Why Explicit Encoding Matters

If you omit the encoding, Python may use a platform-dependent default. On some systems that default happens to be UTF-8, but relying on that makes bugs intermittent and environment-specific.

The result can be:

• decoding errors on one machine but not another
• corrupted characters after writing
• files that appear fine in one editor and broken in another

Explicit encoding removes that ambiguity.

Handling Invalid Data

Sometimes the file is supposed to be UTF-8 but contains bad bytes. In that case, you can decide how strict you want Python to be with the errors parameter.

with open("example.txt", "r", encoding="utf-8", errors="replace") as f:
    content = f.read()

Common choices are:

• 'errors="strict" for fail-fast behavior'
• 'errors="replace" to substitute invalid bytes with a replacement character'
• 'errors="ignore" to skip invalid bytes, though that can silently lose data'

For debugging or data-cleaning work, replace is often safer than ignore because it preserves the fact that something went wrong.

Using pathlib

pathlib provides the same functionality with a more modern file API.

1from pathlib import Path
2
3path = Path("example.txt")
4path.write_text("naïve façade", encoding="utf-8")
5
6content = path.read_text(encoding="utf-8")
7print(content)

This is especially nice in scripts that already use Path objects heavily.

Text Mode vs Binary Mode

If you open a file in binary mode, Python gives you bytes and does not perform any encoding or decoding for you.

1data = "مرحبا".encode("utf-8")
2
3with open("example.bin", "wb") as f:
4    f.write(data)
5
6with open("example.bin", "rb") as f:
7    raw = f.read()
8
9print(raw.decode("utf-8"))

Binary mode is appropriate when you want direct control over the byte representation. Text mode with encoding="utf-8" is simpler when the file is plain text.

Common Pitfalls

The biggest pitfall is assuming Unicode and UTF-8 are the same thing. Unicode is the character model; UTF-8 is one byte encoding for storing Unicode text.

Another common mistake is omitting the encoding argument and hoping the system default is correct. That can work on your machine and fail elsewhere with no code changes.

Developers also misuse errors="ignore" as a quick fix. It hides encoding problems by dropping data, which is often worse than getting a visible exception.

Summary

• In Python 3, strings are Unicode, but files store bytes.
• Use open(..., encoding="utf-8") to read and write UTF-8 text explicitly.
• Add an errors strategy when the input may contain invalid bytes.
• Use binary mode only when you want manual control over encoding and decoding.
• Explicit UTF-8 handling is the simplest way to avoid cross-platform text bugs.