Using pickle.dump - TypeError must be str, not bytes

pickle

python

typeerror

serialization

debugging

Using pickle.dump - TypeError must be str, not bytes

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Start Practicing Learn More

Introduction

The error TypeError: must be str, not bytes when using pickle.dump() means you opened the file in text mode ('w') instead of binary mode ('wb'). Pickle serializes Python objects to a byte stream, so the output file must be opened in binary mode. This is a Python 3 issue — Python 2 allowed text mode for pickle because it did not distinguish between str and bytes.

The Error

python

1import pickle
2
3data = {'name': 'Alice', 'scores': [95, 87, 92]}
4
5# WRONG — text mode
6with open('data.pkl', 'w') as f:
7    pickle.dump(data, f)
8# TypeError: must be str, not bytes

pickle.dump() writes bytes, but the file opened with 'w' expects strings. The fix is one character.

The Fix

python

1import pickle
2
3data = {'name': 'Alice', 'scores': [95, 87, 92]}
4
5# CORRECT — binary mode ('wb')
6with open('data.pkl', 'wb') as f:
7    pickle.dump(data, f)
8
9# Loading also requires binary mode ('rb')
10with open('data.pkl', 'rb') as f:
11    loaded = pickle.load(f)
12
13print(loaded)  # {'name': 'Alice', 'scores': [95, 87, 92]}

The b in 'wb' and 'rb' tells Python to handle the file as binary (bytes), not text (strings).

File Mode Reference

Mode	Type	Use
`'w'`	Text (str)	Writing text files (.txt, .csv)
`'wb'`	Binary (bytes)	Writing pickle, images, protobuf
`'r'`	Text (str)	Reading text files
`'rb'`	Binary (bytes)	Reading pickle, images
`'ab'`	Binary append	Appending to binary files

Same Error with pickle.dumps() to File

python

1import pickle
2
3data = [1, 2, 3]
4
5# pickle.dumps() returns bytes
6serialized = pickle.dumps(data)
7print(type(serialized))  # <class 'bytes'>
8
9# WRONG — writing bytes to a text file
10with open('data.pkl', 'w') as f:
11    f.write(serialized)
12# TypeError: write() argument must be str, not bytes
13
14# CORRECT
15with open('data.pkl', 'wb') as f:
16    f.write(serialized)

Using pickle with io.BytesIO

For in-memory binary streams:

python

1import pickle
2import io
3
4data = {'key': 'value'}
5
6# Write to in-memory buffer
7buffer = io.BytesIO()
8pickle.dump(data, buffer)
9
10# Read back
11buffer.seek(0)
12loaded = pickle.load(buffer)
13print(loaded)  # {'key': 'value'}
14
15# WRONG — using StringIO
16import io
17text_buffer = io.StringIO()
18pickle.dump(data, text_buffer)
19# TypeError: must be str, not bytes

pickle requires BytesIO, not StringIO.

Multiple Objects in One File

python

1import pickle
2
3# Dump multiple objects
4with open('multi.pkl', 'wb') as f:
5    pickle.dump([1, 2, 3], f)
6    pickle.dump({'a': 1}, f)
7    pickle.dump('hello', f)
8
9# Load them in order
10with open('multi.pkl', 'rb') as f:
11    list_data = pickle.load(f)
12    dict_data = pickle.load(f)
13    str_data = pickle.load(f)
14
15print(list_data, dict_data, str_data)
16# [1, 2, 3] {'a': 1} hello

Pickle Protocol Versions

python

1import pickle
2
3data = {'key': 'value'}
4
5# Specify protocol version
6with open('data.pkl', 'wb') as f:
7    pickle.dump(data, f, protocol=pickle.HIGHEST_PROTOCOL)
8
9# Default protocol varies by Python version:
10# Python 3.8+: protocol 5 (default)
11# Python 3.4-3.7: protocol 4
12# Python 3.0-3.3: protocol 3
13
14# For compatibility with Python 2:
15with open('data.pkl', 'wb') as f:
16    pickle.dump(data, f, protocol=2)

Higher protocols are faster and support more types but are not backward-compatible.

Alternatives to Pickle

python

1import json
2
3data = {'name': 'Alice', 'scores': [95, 87, 92]}
4
5# JSON — text-based, human-readable, cross-language
6with open('data.json', 'w') as f:  # Text mode is correct for JSON
7    json.dump(data, f)
8
9with open('data.json', 'r') as f:
10    loaded = json.load(f)

python

1import joblib
2
3# joblib — optimized for NumPy arrays, uses binary mode internally
4import numpy as np
5data = np.random.randn(1000, 1000)
6
7joblib.dump(data, 'data.joblib')
8loaded = joblib.load('data.joblib')

Python 2 vs Python 3

python

1# Python 2: pickle worked with text mode (str was bytes)
2# Python 2 code:
3with open('data.pkl', 'w') as f:  # Worked in Python 2
4    pickle.dump(data, f)
5
6# Python 3: str and bytes are different types
7# This MUST be binary mode in Python 3
8with open('data.pkl', 'wb') as f:
9    pickle.dump(data, f)

When migrating Python 2 code to Python 3, add b to every open() call used with pickle.

Common Pitfalls

Forgetting 'b' in the file mode: The most common cause. 'w' vs 'wb' is a single character difference that causes the error. Always use 'wb' for writing and 'rb' for reading pickle files.
Using open() without context manager: f = open('data.pkl', 'wb') without with risks leaving the file open on exceptions. Always use with open(...) as f:.
Mixing pickle.dump and pickle.dumps: dump() writes to a file object. dumps() returns bytes. Using dumps() and then writing to a text file causes the same error.
Security risk: pickle.load() can execute arbitrary code. Never unpickle data from untrusted sources. Use json for data from external systems.
Loading pickle across Python versions: Pickles created with a higher protocol cannot be loaded by older Python versions. Specify protocol=2 for maximum compatibility.

Summary

Open files in binary mode ('wb'/'rb') for pickle — 'w'/'r' causes TypeError
pickle.dump(obj, file) writes to a binary file; pickle.dumps(obj) returns bytes
Use io.BytesIO for in-memory pickle operations, not io.StringIO
Use json instead of pickle when you need human-readable or cross-language serialization
Never pickle.load() untrusted data — it can execute arbitrary code
Specify protocol=2 when sharing pickles across Python versions