Time Series Analysis
Unevenly Spaced Data
Pandas
Statsmodels
Data Science

Time Series Analysis - unevenly spaced measures - pandas statsmodels

ML System Design practice on Codemia

Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.

Practice ML system design

Introduction

Time series tools in pandas and statsmodels work best when observations are aligned to a regular frequency. Real data often is not. Sensor outages, event-driven measurements, and manual entry can all produce uneven spacing, and that affects both plotting and modeling. The practical workflow is to acknowledge the irregularity first, then decide whether to preserve it, interpolate it, or resample it before fitting any model.

Start by Making the Time Index Explicit

Uneven spacing is easiest to reason about once the timestamps are a proper datetime index.

python
1import pandas as pd
2
3df = pd.DataFrame(
4    {
5        "timestamp": [
6            "2025-01-01 09:00",
7            "2025-01-01 09:07",
8            "2025-01-01 09:31",
9            "2025-01-01 10:02",
10        ],
11        "value": [10.0, 10.8, 11.6, 11.0],
12    }
13)
14
15df["timestamp"] = pd.to_datetime(df["timestamp"])
16df = df.set_index("timestamp").sort_index()
17
18print(df)

At this point, you can already inspect the spacing:

python
print(df.index.to_series().diff())

That quick check is important because many later modeling decisions depend on whether the irregularity is small, large, or structurally meaningful.

Decide Whether to Keep Irregular Timing or Regularize It

Not every uneven series should be forced onto a regular grid. If timestamps are event-based, the gaps may carry real meaning. But many forecasting and decomposition tools in statsmodels assume regular spacing, so regularization is often necessary.

A common approach is to resample to a chosen frequency and then fill gaps deliberately:

python
regular = df.resample("5min").mean()
print(regular.head(10))

This creates a regular time index with missing rows where no observation existed. You now have choices:

  • leave gaps as missing values for diagnostics
  • interpolate if the process is continuous enough
  • forward-fill if the value is stateful and should persist until the next update

Linear interpolation example:

python
regular_interp = regular.interpolate(method="time")
print(regular_interp.head(10))

That is often a reasonable default for smoothly changing measurements, but it is a modeling assumption, not a neutral transformation.

Use statsmodels Only After the Index Semantics Are Clear

Many statsmodels time-series functions expect fixed-frequency data or behave more predictably when the series is regularized. For example, a basic autoregressive workflow becomes much simpler after resampling:

python
1from statsmodels.tsa.ar_model import AutoReg
2
3series = regular_interp["value"].dropna()
4model = AutoReg(series, lags=2, old_names=False)
5result = model.fit()
6
7print(result.summary())

This works because the series now has a coherent notion of "previous step." On an uneven series, "two lags ago" may represent wildly different elapsed time between observations, which changes the interpretation.

If the data is truly irregular and the timing itself matters, regular AR-style models can be misleading. In that case, consider feature engineering with elapsed-time deltas or methods built for irregular events rather than pretending the sample clock is uniform.

Plot the Irregularity Before Modeling It Away

A simple visualization often tells you whether resampling is harmless or dangerous.

python
1import matplotlib.pyplot as plt
2
3fig, ax = plt.subplots(figsize=(8, 4))
4ax.plot(df.index, df["value"], marker="o", label="original")
5ax.plot(regular_interp.index, regular_interp["value"], alpha=0.7, label="resampled/interpolated")
6ax.legend()
7plt.show()

If the resampled line invents long smooth regions across large gaps, that is a warning sign. Interpolation can make plots look tidy while quietly adding synthetic structure the original data never contained.

Choose Filling Rules Based on the Domain

This is where most uneven-series mistakes happen. Different data types want different gap policies:

  • temperature or pressure may tolerate interpolation
  • inventory counts may want forward-fill
  • event counts may want zero-fill only if the absence of records truly means zero activity

For example, forward-fill:

python
stateful = regular.ffill()

Zero-fill:

python
count_series = regular.fillna(0)

Neither is universally correct. The key is that the fill method should represent the underlying process, not just satisfy the model input requirements.

Common Pitfalls

  • Feeding irregular data straight into regular-time models without checking the implied assumptions.
  • Resampling and interpolating before first visualizing how large the gaps actually are.
  • Using a fill method such as forward-fill or zero-fill without considering domain semantics.
  • Treating interpolated observations as if they were original measurements.
  • Ignoring the time-delta structure when the irregular timing itself contains information.

Summary

  • Unevenly spaced measures need explicit handling before most time-series models behave sensibly.
  • Use a datetime index and inspect the time gaps first.
  • Resample only when a regular frequency is appropriate for the analysis.
  • Choose interpolation or filling rules based on the process being measured, not convenience alone.
  • In statsmodels, regularized data is often required, but regularization should be treated as a modeling decision, not just preprocessing boilerplate.

Related reading
Free course
Beginner
7 lessons
2 hours
Tackling System Design Interview Problems

A short course that equips you with the skills to approach system design interviews methodically.

Start the free course
Track what you have practised

A free account saves your progress, solutions and study plan across every problem on Codemia.

ML System Design practice on Codemia

Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.

Practice ML system design

All Rights Reserved.