Introduction

Python has automatic memory management, but long running applications can still show memory growth that looks like a leak. Sometimes the problem is a true leak, and sometimes it is retained objects, caches, or allocator behavior. A practical debugging approach combines measurement, object tracking, and controlled reproduction.

Understand What Counts as a Leak

In Python, memory issues typically fall into three categories:

• real leaks from lingering references that should be released
• intentional retention such as caches that never evict
• memory fragmentation where RSS stays high even after objects are freed

You need to identify which category applies before applying fixes.

Start with Reproducible Measurement

Use process memory sampling in a repeatable workload loop.

1import os
2import psutil
3import time
4
5process = psutil.Process(os.getpid())
6
7for i in range(5):
8    data = [str(x) for x in range(200_000)]
9    del data
10    rss_mb = process.memory_info().rss / (1024 * 1024)
11    print(f'iteration {i} rss_mb={rss_mb:.2f}')
12    time.sleep(0.5)

This gives a baseline trend before deeper inspection.

Track Allocations with tracemalloc

tracemalloc helps locate lines responsible for growth.

1import tracemalloc
2
3tracemalloc.start()
4
5leaky = []
6for i in range(10000):
7    leaky.append('x' * 1000)
8
9snapshot = tracemalloc.take_snapshot()
10for stat in snapshot.statistics('lineno')[:5]:
11    print(stat)

Compare snapshots over time to confirm whether the same code path keeps allocating.

Find Unexpected Reference Chains

If objects should be freed but remain alive, inspect references.

1import gc
2import objgraph
3
4class Node:
5    def __init__(self, name):
6        self.name = name
7
8nodes = [Node(str(i)) for i in range(1000)]
9del nodes
10
11gc.collect()
12print('Node objects:', objgraph.count('Node'))

If count stays high, use object graph back references to find retention roots.

Common Leak Sources in Python Services

Frequent causes include:

• global lists or dictionaries that grow without bounds
• unbounded LRU style caches
• event listeners never deregistered
• large closures capturing request data
• cyclic references with external resources not closed

Many cases are application retention bugs, not interpreter defects.

Practical Fix Patterns

Bound Caches

Use size limited caches.

1from functools import lru_cache
2
3@lru_cache(maxsize=1024)
4def parse_user(user_id: int):
5    return f'user-{user_id}'

Release Resource Handles

Use context managers for files, sockets, and DB connections.

with open('app.log', 'w', encoding='utf-8') as f:
    f.write('hello')

Avoid Accidental Globals

Keep per request data in function scope and clear containers when done.

Distinguish RSS from Python Heap

Process RSS may remain high due to allocator behavior even when Python objects are freed. Focus on object counts and allocation snapshots, not RSS alone.

For containerized workloads, combine Python metrics with process and cgroup metrics to avoid wrong conclusions.

Add Leak Tests to CI for Long Running Jobs

For batch workers, run repeated job loops in tests and assert memory slope stays within threshold.

def memory_growth_allowed(samples, max_growth_mb=50):
    return (max(samples) - min(samples)) <= max_growth_mb

Automated checks catch regressions early when code changes introduce retention paths.

Common Pitfalls

A common pitfall is calling gc.collect repeatedly in production hoping to solve growth. This can hide symptoms but rarely fixes root references.

Another issue is treating every high RSS chart as leak evidence without object-level confirmation.

A third issue is unbounded debugging logs or telemetry buffers that grow over time and mimic memory leaks.

Teams also skip load profiles and test only short runs, missing leaks that appear after hours.

Summary

• Python memory growth can come from leaks, retention, or allocator behavior
• Start with repeatable measurement and then inspect allocations with tooling
• Use tracemalloc and reference graph tools to locate retention roots
• Fix common causes such as unbounded caches and long lived references
• Add long run memory regression checks to prevent recurring issues