Sorting algorithms
Statistical distribution
Data sorting
Algorithm optimization
Data analysis

Sorting algorithms for data of known statistical distribution?

ML System Design practice on Codemia

Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.

Practice ML system design

Introduction

If you know something about the distribution of your input data, you may be able to sort faster than a generic comparison sort. The key idea is that distribution-aware algorithms trade strict generality for assumptions about value ranges, density, or digit structure.

Why Distribution Matters

Comparison sorts such as quicksort, mergesort, and heapsort are excellent defaults, but they are bounded by the usual comparison-based lower limit of O(n log n) in the general case. If your data values come from a known range or follow a predictable layout, non-comparison techniques can do better.

Typical examples include:

  • integers in a small bounded range
  • floating-point values roughly uniform over a fixed interval
  • fixed-width keys such as IDs or timestamps

The point is not that the data must match a textbook probability curve exactly. The useful question is whether you can partition or index the values cheaply.

Counting Sort for Bounded Integers

Counting sort works well when keys are integers and the range is not too large compared with the number of items. Instead of comparing elements against each other, it counts how many times each value appears.

python
1def counting_sort(values, max_value):
2    counts = [0] * (max_value + 1)
3
4    for value in values:
5        counts[value] += 1
6
7    result = []
8    for value, count in enumerate(counts):
9        result.extend([value] * count)
10
11    return result
12
13
14nums = [4, 2, 2, 8, 3, 3, 1]
15print(counting_sort(nums, max(nums)))

This is O(n + k), where k is the size of the value range. That is excellent when k is small, but wasteful when values are sparse across a huge domain.

Bucket Sort for Roughly Uniform Data

Bucket sort is a strong fit when values are spread somewhat evenly across a known interval. The algorithm places each value into a bucket, sorts each bucket locally, and concatenates the results.

python
1def bucket_sort(values, bucket_count=5):
2    if not values:
3        return []
4
5    min_value = min(values)
6    max_value = max(values)
7    if min_value == max_value:
8        return values[:]
9
10    buckets = [[] for _ in range(bucket_count)]
11    scale = (bucket_count - 1) / (max_value - min_value)
12
13    for value in values:
14        index = int((value - min_value) * scale)
15        buckets[index].append(value)
16
17    result = []
18    for bucket in buckets:
19        result.extend(sorted(bucket))
20
21    return result
22
23
24data = [0.42, 0.32, 0.23, 0.52, 0.25, 0.47, 0.51]
25print(bucket_sort(data))

When the distribution is close to uniform, buckets stay small and local sorting is cheap. If the distribution is highly skewed, one bucket can become overloaded and the advantage disappears.

Radix Sort for Fixed-Width Keys

Radix sort is useful when keys can be processed digit by digit, such as integers, zero-padded strings, or fixed-length identifiers. It avoids direct element comparisons and instead performs stable passes over each digit position.

python
1def radix_sort(values):
2    if not values:
3        return []
4
5    output = values[:]
6    exp = 1
7    max_value = max(output)
8
9    while max_value // exp > 0:
10        buckets = [[] for _ in range(10)]
11        for number in output:
12            digit = (number // exp) % 10
13            buckets[digit].append(number)
14
15        output = [number for bucket in buckets for number in bucket]
16        exp *= 10
17
18    return output
19
20
21nums = [170, 45, 75, 90, 802, 24, 2, 66]
22print(radix_sort(nums))

For machine data with predictable numeric widths, radix sort can be very fast and stable.

When Generic Sorts Still Win

Sometimes the distribution is "known" only in a vague sense. For example, you may know the data is normally distributed, but not have a useful bounded mapping or bucket strategy. In that case, a high-quality built-in sort is often still the right engineering choice. Most standard library sorts are heavily optimized, and they do not fall apart when assumptions are violated.

A practical approach is:

  • use counting sort for small bounded integer ranges
  • use bucket sort for nearly uniform values in a known interval
  • use radix sort for structured fixed-width keys
  • use built-in comparison sorts when assumptions are weak or data is mixed

Common Pitfalls

  • Choosing counting sort when the numeric range is huge. Memory usage can dominate the runtime.
  • Assuming bucket sort is always fast. It depends heavily on buckets staying balanced.
  • Forgetting that radix sort usually targets specific key representations, not arbitrary comparison logic.
  • Over-optimizing for a distribution that is only true in tests, not in production traffic.

Summary

  • Known data distribution can justify faster-than-comparison sorting strategies.
  • Counting sort is ideal for dense bounded integer ranges.
  • Bucket sort is effective for roughly uniform data in a known interval.
  • Radix sort works well for fixed-width numeric or digit-based keys.
  • When assumptions are uncertain, a built-in general-purpose sort is usually safer.

Related reading
Course
Intermediate
27 lessons
15 hours
DSA Fundamentals

Master algorithmic patterns and data structures through hands-on LeetCode-style problems - from arrays and hashing to dynamic programming and advanced graphs.

View the course
Track what you have practised

A free account saves your progress, solutions and study plan across every problem on Codemia.

ML System Design practice on Codemia

Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.

Practice ML system design