C# benchmarking
code optimization
performance analysis
software development
code improvement

Benchmarking small code samples in C, can this implementation be improved?

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Introduction

Benchmarking tiny C# code samples is harder than it looks. For short operations, measurement overhead, JIT compilation, dead-code elimination, and garbage collection can easily dominate the numbers. A good benchmark isolates the work, repeats it many times, and uses a framework that understands how .NET actually runs code.

Why Naive Timing Is Misleading

A common first attempt is to wrap a method call in Stopwatch. That can be useful for coarse application timing, but it is usually not reliable for microbenchmarks.

csharp
1using System;
2using System.Diagnostics;
3
4class Program
5{
6    static int SumLoop(int n)
7    {
8        int sum = 0;
9        for (int i = 0; i < n; i++)
10        {
11            sum += i;
12        }
13        return sum;
14    }
15
16    static void Main()
17    {
18        var sw = Stopwatch.StartNew();
19        int result = SumLoop(1000);
20        sw.Stop();
21        Console.WriteLine($"Result={result}, Elapsed={sw.ElapsedTicks} ticks");
22    }
23}

This number mixes together JIT cost, timer overhead, process state, and the work itself. If the code is very fast, the measurement is mostly noise.

Use BenchmarkDotNet for Small Samples

For microbenchmarks, BenchmarkDotNet is the standard tool in the .NET ecosystem because it handles warm-up, repeated runs, statistical summaries, and environment reporting.

csharp
1using BenchmarkDotNet.Attributes;
2using BenchmarkDotNet.Running;
3
4[MemoryDiagnoser]
5public class SumBenchmarks
6{
7    [Params(100, 1000, 10000)]
8    public int N;
9
10    [Benchmark(Baseline = true)]
11    public int ForLoop()
12    {
13        int sum = 0;
14        for (int i = 0; i < N; i++)
15        {
16            sum += i;
17        }
18        return sum;
19    }
20
21    [Benchmark]
22    public int LinqSum()
23    {
24        return Enumerable.Range(0, N).Sum();
25    }
26}
27
28public class Program
29{
30    public static void Main(string[] args)
31    {
32        BenchmarkRunner.Run<SumBenchmarks>();
33    }
34}

This tells you much more than a raw elapsed time. You get mean time, error, standard deviation, memory allocation, and the runtime configuration used during the benchmark.

Benchmark the Right Thing

A benchmark should focus on one question. For example:

  • is for faster than LINQ for this workload
  • does a new allocation appear after a refactor
  • how does runtime change when the input grows

If a single benchmark mixes setup, data generation, logging, and the actual code under test, the result becomes hard to interpret.

A better pattern is to move setup into a dedicated setup method and keep the benchmark body as small as possible.

csharp
1using BenchmarkDotNet.Attributes;
2
3public class SearchBenchmarks
4{
5    private int[] _data = Array.Empty<int>();
6
7    [GlobalSetup]
8    public void Setup()
9    {
10        _data = Enumerable.Range(0, 10000).ToArray();
11    }
12
13    [Benchmark]
14    public bool ArrayContains()
15    {
16        return Array.IndexOf(_data, 9999) >= 0;
17    }
18}

This separates test-data construction from the operation being measured.

Avoid Common Benchmarking Traps

One trap is benchmarking Debug builds. Debug mode changes optimization behavior and often makes results meaningless for real performance decisions. Use Release mode.

Another trap is letting the compiler optimize away unused results. If the return value is not consumed, the optimizer may remove work you thought you were measuring. BenchmarkDotNet helps protect against that, but the general rule still matters.

A third trap is benchmarking code with unrealistic input. If production inputs are large, random, sparse, or already sorted, your benchmark should reflect that. Tiny toy inputs often tell the wrong story.

Read the Results Correctly

Do not focus only on the mean. Standard deviation, allocation size, and the relative gap between methods often matter more than one absolute number.

For example, a method that is three nanoseconds faster but allocates more memory might still be a worse choice in a hot loop. Similarly, if the difference is smaller than the measurement noise, the benchmark is not strong evidence that one approach is truly better.

Also remember that microbenchmark wins do not always translate into end-to-end application wins. Benchmark the small piece, but validate the change in the real workload too.

Common Pitfalls

The most common mistake is using Stopwatch once and treating the result as authoritative. For small samples, that is usually just noise.

Another problem is benchmarking code in Debug mode or under a debugger. That changes execution characteristics significantly.

Developers also often benchmark setup work instead of the core operation. If data creation dominates the measurement, the benchmark is answering the wrong question.

Finally, do not over-optimize a trivial method in isolation when the application bottleneck is elsewhere. Benchmarking is only useful when it supports a real performance decision.

Summary

  • Use BenchmarkDotNet for small C# benchmarks instead of hand-rolled timing.
  • Keep setup separate from the benchmarked operation.
  • Run Release builds and avoid debugger-influenced measurements.
  • Benchmark realistic inputs and inspect allocations as well as time.
  • Treat microbenchmark results as evidence, not as the whole performance story.

Course illustration
Course illustration

All Rights Reserved.