Introduction

String similarity is not one problem with one metric. Matching person names, deduplicating product titles, and comparing full documents all reward different notions of "similar." Python has good tools for edit distance, token overlap, phonetic matching, and vector-space similarity, but choosing the wrong metric can be worse than using none at all.

Edit Distance for Small Character-Level Differences

If you care about typos, insertions, deletions, or substitutions, start with edit distance. A practical Python option is rapidfuzz, which is fast and easy to use.

1from rapidfuzz import fuzz
2
3s1 = "kitten"
4s2 = "sitting"
5
6print(fuzz.ratio(s1, s2))
7print(fuzz.partial_ratio("new york city", "york"))

Use character-level metrics when:

• the strings are short
• order matters
• spelling mistakes are common

This works well for names, SKUs, or lightly noisy labels. It is less useful for long documents where token structure matters more than small character edits.

Token-Based Similarity for Word Reordering

If the same words appear in different order, token-based metrics are usually better than raw edit distance.

1from rapidfuzz import fuzz
2
3a = "apple iphone 14 pro max"
4b = "iphone apple pro max 14"
5
6print(fuzz.token_sort_ratio(a, b))
7print(fuzz.token_set_ratio(a, b))

token_sort_ratio helps when the words are the same but reordered. token_set_ratio is useful when one string contains most of the other plus extra words.

These metrics are common in search normalization and catalog matching.

Jaccard Similarity for Set Overlap

When you only care about overlap between token sets, Jaccard similarity is simple and interpretable.

1def jaccard_similarity(a: str, b: str) -> float:
2    sa = set(a.lower().split())
3    sb = set(b.lower().split())
4    if not sa and not sb:
5        return 1.0
6    return len(sa & sb) / len(sa | sb)
7
8
9print(jaccard_similarity("red apple fresh", "fresh red apple"))
10print(jaccard_similarity("red apple fresh", "banana yellow"))

Jaccard ignores repeated words and character-level typos. That makes it poor for misspellings but useful for rough token overlap.

Cosine Similarity for Longer Text

For sentences or documents, a vector-space approach is often more useful. A simple baseline is TF-IDF plus cosine similarity.

1from sklearn.feature_extraction.text import TfidfVectorizer
2from sklearn.metrics.pairwise import cosine_similarity
3
4docs = [
5    "python string similarity metrics",
6    "measuring text similarity in python",
7]
8
9vec = TfidfVectorizer()
10X = vec.fit_transform(docs)
11
12score = cosine_similarity(X[0], X[1])[0, 0]
13print(score)

This is better for document-level comparison than plain edit distance because it focuses on shared terms and their relative importance.

Phonetic Matching for Names

If your data contains names that sound similar but are spelled differently, phonetic methods can help. They are not universal, but they can be useful in address books, genealogy, or customer records.

1from jellyfish import soundex
2
3print(soundex("Smith"))
4print(soundex("Smyth"))

The point is not that phonetic codes are perfect. It is that they solve a different problem than edit distance does.

Choosing the Right Metric

A useful decision guide:

• typo-heavy short strings: Levenshtein-style metrics
• reordered product titles: token sort or token set metrics
• word-overlap heuristics: Jaccard
• document similarity: TF-IDF plus cosine
• names with pronunciation variation: phonetic matching

In real systems, teams often combine two stages. For example, candidate generation might use TF-IDF, and final ranking might use a token-based fuzzy score.

Common Pitfalls

• Using one metric for every text-matching problem.
• Applying character edit distance to long documents.
• Ignoring normalization such as lowercasing, punctuation removal, or Unicode cleanup.
• Treating similarity scores from different metrics as directly comparable.
• Forgetting that domain-specific rules often matter more than the metric itself.

Summary

• String similarity depends on what kind of difference you care about.
• Edit distance is good for typos and short strings.
• Token-based metrics handle reordered phrases better.
• Cosine similarity is a stronger baseline for longer text.
• Phonetic methods are useful for name-like data.