Introduction

In Python, writing Unicode text to a file is usually simple: open the file in text mode and specify an encoding such as UTF-8. The main problems come from relying on platform defaults, mixing text and bytes APIs, or reading the file later with a different encoding than the one used to write it.

Open the file in text mode with an explicit encoding

The safest default is UTF-8:

1text = "Hello, 世界 👋"
2
3with open("output.txt", "w", encoding="utf-8") as f:
4    f.write(text)

That is the core answer. Python will encode the Unicode string into UTF-8 bytes before writing it to disk.

Using an explicit encoding matters because default encodings vary across platforms and environments. A script that works on one machine may produce mojibake or raise errors on another if the encoding is left implicit.

Read it back with the same encoding

Writing succeeds only half the time if later code reads the file with the wrong encoding. Pair the read side with the same explicit encoding:

1with open("output.txt", "r", encoding="utf-8") as f:
2    content = f.read()
3
4print(content)

This is especially important in command-line tools, data pipelines, and test fixtures where files may be opened on different machines.

Understand the difference between text and bytes

Python text files expect str, not raw bytes. If you already have bytes, open the file in binary mode instead:

1data = "Hello, 世界 👋".encode("utf-8")
2
3with open("output.bin", "wb") as f:
4    f.write(data)

Likewise, do not manually encode a string and then pass it to a text-mode file object. That mixes the two layers incorrectly.

The practical rule is:

• use text mode plus encoding= when working with str
• use binary mode when working with bytes

pathlib works the same way

If you prefer pathlib, the same rule applies:

1from pathlib import Path
2
3path = Path("output.txt")
4path.write_text("Café — Привет — こんにちは", encoding="utf-8")
5
6content = path.read_text(encoding="utf-8")
7print(content)

This can be cleaner in scripts that already use Path objects for filesystem work.

Handle replacement and error strategies deliberately

Sometimes you are not sure whether all characters can be represented in the target encoding. In that case, Python lets you choose an error strategy:

1text = "Hello, 世界 👋"
2
3with open("ascii-output.txt", "w", encoding="ascii", errors="replace") as f:
4    f.write(text)

That will replace characters ASCII cannot represent. Usually, though, the better answer is not to degrade the data at all. It is to choose UTF-8 unless you have a real external constraint.

Know when a BOM matters

UTF-8 usually does not need a byte order mark. But some older Windows tools expect one. If you must interoperate with such tools, Python provides utf-8-sig:

with open("output-with-bom.txt", "w", encoding="utf-8-sig") as f:
    f.write("Hello, 世界")

That is not the normal default for modern systems, but it can be useful when exchanging files with legacy software.

Common Pitfalls

The most common mistake is writing Unicode text without specifying encoding, then being surprised when the file looks wrong on another machine.

Another common issue is mixing text and binary APIs, such as encoding a string manually and then writing it through a text-mode file.

People also forget that reading must use the same encoding assumptions as writing. A correctly written UTF-8 file can still look broken if it is read as something else.

Finally, if a downstream system requires ASCII or another restricted encoding, do not ignore the data-loss implications. Decide whether replacement, escaping, or a different transport format is acceptable.

Summary

• In Python, write Unicode text with open(..., encoding="utf-8").
• Read it back with the same explicit encoding.
• Use text mode for str and binary mode for bytes.
• Prefer UTF-8 unless you have a specific compatibility constraint.
• Be deliberate about error handling and BOM usage when interoperating with older tools.