Keras LSTM predicted timeseries squashed and shifted

LSTM

Keras

timeseries prediction

data transformation

machine learning

Keras LSTM predicted timeseries squashed and shifted

ML System Design practice on Codemia

Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.

Practice ML system design

Deep Learning TensorFlow & Keras Machine Learning

Introduction

When LSTM predictions appear "squashed" (compressed into a narrow range) and "shifted" (lagging behind the actual values), the root causes are almost always improper data scaling, the model learning to predict the previous timestep, or insufficient model complexity. The squashed output means the model is predicting near the mean of the target values, and the shift means the model has learned that the best low-loss strategy is to output the last seen input value. Both issues indicate the model is not learning meaningful temporal patterns.

The Symptom

python

# Predictions look like a smoothed, delayed copy of the actual values
# Actual:     [10, 15, 8, 20, 12, 25, 9]
# Predicted:  [12, 11, 13, 10, 17, 13, 21]  <- shifted right, compressed range

The predictions track the general trend but are always one or more steps behind, and the peaks and valleys are much smaller than the actual data.

Cause 1: Model Predicting the Previous Value

The easiest way for an LSTM to minimize loss on a time series is to output the previous input value. This produces a prediction that looks like the actual data shifted by one timestep:

python

1import numpy as np
2from tensorflow.keras.models import Sequential
3from tensorflow.keras.layers import LSTM, Dense
4
5# Create sequences: X[t] = [values at t-n:t], y[t] = value at t+1
6def create_sequences(data, seq_length):
7    X, y = [], []
8    for i in range(len(data) - seq_length):
9        X.append(data[i:i + seq_length])
10        y.append(data[i + seq_length])
11    return np.array(X), np.array(y)
12
13# If sequence length is too short (e.g., 1), the model just copies the input
14X, y = create_sequences(data, seq_length=1)
15# X[0] = [value_at_t0], y[0] = value_at_t1
16# Model learns: predict(X) ≈ X (copy the input)

Fix: Use a longer sequence length so the model has enough context to learn patterns, not just copy:

python

# Use 30-60 timesteps for daily data
X, y = create_sequences(data, seq_length=30)

Cause 2: Improper Scaling

LSTMs are sensitive to input scale. Without normalization, the model struggles to learn and defaults to predicting near the mean:

python

1from sklearn.preprocessing import MinMaxScaler
2
3# BAD: raw values with large range
4# data = [100, 5000, 250, 8000, ...]  <- LSTM struggles
5
6# GOOD: scale to [0, 1]
7scaler = MinMaxScaler(feature_range=(0, 1))
8data_scaled = scaler.fit_transform(data.reshape(-1, 1))
9
10# Create sequences from scaled data
11X, y = create_sequences(data_scaled, seq_length=30)
12
13# After prediction, inverse transform to original scale
14predictions_scaled = model.predict(X_test)
15predictions = scaler.inverse_transform(predictions_scaled)

A common mistake is fitting the scaler on the entire dataset (including test data), which leaks future information:

python

1# WRONG: fit on all data (data leakage)
2scaler.fit(all_data)
3
4# CORRECT: fit only on training data
5scaler.fit(train_data)
6train_scaled = scaler.transform(train_data)
7test_scaled = scaler.transform(test_data)

Cause 3: Insufficient Model Complexity

A single LSTM layer with few units may not have enough capacity:

python

1# Too simple — predicts the mean
2model = Sequential([
3    LSTM(10, input_shape=(seq_length, n_features)),
4    Dense(1)
5])
6
7# Better — stacked LSTM with more units
8model = Sequential([
9    LSTM(64, return_sequences=True, input_shape=(seq_length, n_features)),
10    LSTM(32),
11    Dense(1)
12])
13
14model.compile(optimizer='adam', loss='mse')
15model.fit(X_train, y_train, epochs=100, batch_size=32, validation_split=0.2)

Cause 4: Wrong Loss Function or Metric

Using MSE loss causes the model to predict the mean when it cannot learn the pattern:

python

1# MSE penalizes large errors quadratically
2# The safest prediction under MSE is the mean of the target distribution
3
4# Try different losses
5model.compile(optimizer='adam', loss='mae')   # Less sensitive to outliers
6model.compile(optimizer='adam', loss='huber')  # Balanced between MSE and MAE

Cause 5: Incorrect Inverse Transform

Forgetting to reverse the scaling makes predictions look squashed:

python

1# Predictions in [0, 1] range look "squashed" compared to original data
2predictions = model.predict(X_test)
3# predictions: [0.42, 0.45, 0.41, ...]  <- looks squashed!
4
5# Must inverse transform
6predictions_original = scaler.inverse_transform(predictions)
7# predictions_original: [4200, 4500, 4100, ...]  <- correct scale

Correct End-to-End Example

python

1import numpy as np
2import pandas as pd
3from sklearn.preprocessing import MinMaxScaler
4from tensorflow.keras.models import Sequential
5from tensorflow.keras.layers import LSTM, Dense, Dropout
6from tensorflow.keras.callbacks import EarlyStopping
7
8# Load and prepare data
9data = pd.read_csv('timeseries.csv')['value'].values.reshape(-1, 1)
10
11# Split BEFORE scaling
12train_size = int(len(data) * 0.8)
13train_data = data[:train_size]
14test_data = data[train_size:]
15
16# Scale based on training data only
17scaler = MinMaxScaler(feature_range=(0, 1))
18train_scaled = scaler.fit_transform(train_data)
19test_scaled = scaler.transform(test_data)
20
21# Create sequences
22seq_length = 30
23X_train, y_train = create_sequences(train_scaled, seq_length)
24X_test, y_test = create_sequences(test_scaled, seq_length)
25
26# Reshape for LSTM: (samples, timesteps, features)
27X_train = X_train.reshape(X_train.shape[0], X_train.shape[1], 1)
28X_test = X_test.reshape(X_test.shape[0], X_test.shape[1], 1)
29
30# Build model
31model = Sequential([
32    LSTM(64, return_sequences=True, input_shape=(seq_length, 1)),
33    Dropout(0.2),
34    LSTM(32),
35    Dropout(0.2),
36    Dense(1)
37])
38
39model.compile(optimizer='adam', loss='mse')
40
41# Train with early stopping
42early_stop = EarlyStopping(monitor='val_loss', patience=10, restore_best_weights=True)
43model.fit(X_train, y_train, epochs=100, batch_size=32,
44          validation_split=0.2, callbacks=[early_stop])
45
46# Predict and inverse transform
47predictions = model.predict(X_test)
48predictions = scaler.inverse_transform(predictions)
49actual = scaler.inverse_transform(y_test.reshape(-1, 1))

Diagnosing the Problem

python

1import matplotlib.pyplot as plt
2
3# Plot predictions vs actual
4plt.figure(figsize=(12, 6))
5plt.plot(actual, label='Actual', color='blue')
6plt.plot(predictions, label='Predicted', color='red')
7plt.legend()
8plt.title('LSTM Predictions vs Actual')
9plt.show()
10
11# Check if predictions are just shifted actuals
12correlation = np.corrcoef(actual[1:].flatten(), predictions[:-1].flatten())[0, 1]
13print(f"Correlation with shifted actual: {correlation:.4f}")
14# If > 0.99, the model is just copying the previous value

Common Pitfalls

Data leakage through scaling: Fitting the scaler on the entire dataset (train + test) leaks information about the test set's distribution. Always fit the scaler on training data only and use transform() on test data.
Sequence length too short: With seq_length=1, the LSTM has no temporal context and simply copies the input. Use at least 20-60 timesteps depending on the data's periodicity.
Forgetting inverse transform: Predictions in the scaled range (0 to 1) look squashed when plotted against original data. Always call scaler.inverse_transform() before comparing or plotting.
Not using return_sequences=True for stacked LSTMs: When stacking multiple LSTM layers, all layers except the last must use return_sequences=True to pass the full sequence to the next layer. Without it, only the last timestep is passed.
Evaluating on training data: If predictions look good on training data but squashed on test data, the model is overfitting. Use early stopping, dropout, and a separate validation set to detect overfitting.

Summary

Squashed predictions mean the model predicts near the mean — usually caused by bad scaling or insufficient complexity
Shifted predictions mean the model copies the previous value — increase sequence length and model capacity
Always scale data with MinMaxScaler or StandardScaler, fitting only on training data
Use stacked LSTM layers with dropout for better temporal pattern learning
Always inverse transform predictions before comparing with actual values

Tackling System Design Interview Problems

A short course that equips you with the skills to approach system design interviews methodically.

Start the free course

Track what you have practised

A free account saves your progress, solutions and study plan across every problem on Codemia.

ML System Design practice on Codemia

Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.

Practice ML system design

Keras LSTM predicted timeseries squashed and shifted

ML System Design practice on Codemia

Introduction

The Symptom

Cause 1: Model Predicting the Previous Value

Cause 2: Improper Scaling

Cause 3: Insufficient Model Complexity

Cause 4: Wrong Loss Function or Metric

Cause 5: Incorrect Inverse Transform

Correct End-to-End Example

Diagnosing the Problem

Common Pitfalls

Summary

Related reading

Tackling System Design Interview Problems

Track what you have practised

ML System Design practice on Codemia