Introduction

No — Python dictionaries are not slow for character frequency counting. Dictionary lookups and insertions are O(1) average-case, making them one of the most efficient data structures for this task. A manual dictionary loop, collections.Counter, and defaultdict(int) all use hash tables internally and perform similarly. The fastest built-in option is collections.Counter, which is implemented in C (CPython). For absolute maximum speed on large strings, str.count() per character or NumPy approaches can be marginally faster.

Basic Dictionary Approach

1text = "hello world"
2
3freq = {}
4for char in text:
5    if char in freq:
6        freq[char] += 1
7    else:
8        freq[char] = 1
9
10print(freq)
11# {'h': 1, 'e': 1, 'l': 3, 'o': 2, ' ': 1, 'w': 1, 'r': 1, 'd': 1}

Each in check and [] assignment is O(1) on average. Total time is O(n) where n is the string length.

collections.Counter — Recommended

1from collections import Counter
2
3text = "hello world"
4freq = Counter(text)
5
6print(freq)
7# Counter({'l': 3, 'o': 2, 'h': 1, 'e': 1, ' ': 1, 'w': 1, 'r': 1, 'd': 1})
8
9# Most common characters
10print(freq.most_common(3))
11# [('l', 3), ('o', 2), ('h', 1)]
12
13# Access specific character
14print(freq['l'])  # 3
15print(freq['z'])  # 0 (missing keys return 0, not KeyError)

Counter is a dict subclass with a C-optimized __init__ that counts elements faster than a Python loop.

defaultdict(int) Approach

1from collections import defaultdict
2
3text = "hello world"
4freq = defaultdict(int)
5for char in text:
6    freq[char] += 1
7
8print(dict(freq))
9# {'h': 1, 'e': 1, 'l': 3, 'o': 2, ' ': 1, 'w': 1, 'r': 1, 'd': 1}

defaultdict(int) auto-creates missing keys with value 0, eliminating the if key in dict check.

dict.get() Approach

freq = {}
for char in text:
    freq[char] = freq.get(char, 0) + 1

get(key, default) returns 0 for missing keys. Slightly slower than defaultdict but needs no import.

Performance Benchmarks

1import timeit
2from collections import Counter, defaultdict
3
4text = "a" * 100_000 + "b" * 50_000 + "c" * 30_000  # 180K characters
5
6# 1. collections.Counter (fastest)
7def using_counter():
8    return Counter(text)
9
10# 2. defaultdict(int)
11def using_defaultdict():
12    freq = defaultdict(int)
13    for char in text:
14        freq[char] += 1
15    return freq
16
17# 3. Manual dict with if/else
18def using_dict():
19    freq = {}
20    for char in text:
21        if char in freq:
22            freq[char] += 1
23        else:
24            freq[char] = 1
25    return freq
26
27# 4. dict.get()
28def using_get():
29    freq = {}
30    for char in text:
31        freq[char] = freq.get(char, 0) + 1
32    return freq
33
34# Typical results (CPython 3.11, 180K chars):
35# Counter:      ~3.5 ms
36# defaultdict:  ~8.0 ms
37# dict if/else: ~9.5 ms
38# dict.get():   ~10.0 ms

Counter is ~2-3x faster than manual Python loops because its counting loop runs in C.

Why Dictionaries Are Fast

Python dictionaries use hash tables with:

• O(1) average lookup: Hash the key, jump to the bucket, compare
• O(1) average insertion: Same hash + bucket logic
• Dynamic resizing: The table grows (typically doubles) when 2/3 full
• Open addressing: Collisions are handled by probing, not chaining

For character frequency, the number of unique keys is at most 256 (ASCII) or ~150K (Unicode), so collisions are rare and the hash table stays small.

Alternative: str.count() Per Character

For a known small alphabet:

1import string
2
3text = "hello world"
4
5# Count only lowercase letters
6freq = {char: text.count(char) for char in set(text)}
7# {'h': 1, 'e': 1, 'l': 3, 'o': 2, ' ': 1, 'w': 1, 'r': 1, 'd': 1}

str.count() is C-implemented and fast per call, but iterates the entire string once per unique character. For strings with few unique characters, this can beat Counter. For strings with many unique characters, Counter wins.

NumPy for Very Large Strings

1import numpy as np
2
3text = "hello world" * 100_000
4
5# Convert to byte array and use bincount
6arr = np.frombuffer(text.encode('ascii'), dtype=np.uint8)
7counts = np.bincount(arr, minlength=256)
8
9# Get results as dict
10freq = {chr(i): int(c) for i, c in enumerate(counts) if c > 0}

NumPy's bincount is the fastest option for ASCII strings over ~1MB because it operates on contiguous memory without Python object overhead.

Common Pitfalls

• Assuming dictionaries are slow: Python dicts are hash tables with O(1) operations. They are one of the most optimized data structures in CPython. For frequency counting, the bottleneck is the Python for loop overhead, not the dictionary.
• Using list.count() in a loop: [text.count(c) for c in text] runs count() for every character in the string (including duplicates), making it O(n^2). Always iterate over set(text) or use Counter.
• Forgetting that Counter returns 0 for missing keys: Counter(text)['z'] returns 0, not KeyError. This is different from a regular dict where d['z'] would raise KeyError.
• Sorting by frequency incorrectly: sorted(freq) sorts by keys, not values. Use sorted(freq.items(), key=lambda x: x[1], reverse=True) or Counter.most_common() to sort by frequency.
• Unicode normalization: Characters like 'e' and 'e with accent' are different Unicode code points. If you want accent-insensitive counting, normalize with unicodedata.normalize('NFKD', text) first.

Summary

• Python dictionaries are O(1) for lookup and insertion — they are not slow for frequency counting
• collections.Counter is the fastest and most Pythonic choice (C-optimized counting)
• defaultdict(int) is the second-best option, avoiding explicit key existence checks
• Manual dict with if/else or .get() works but is ~2-3x slower than Counter due to Python loop overhead
• For very large ASCII strings (1MB+), NumPy's bincount is the absolute fastest option