Introduction

When the TensorFlow Object Detection API behaves strangely, the root cause is often not the detector itself but a mismatch somewhere in preprocessing, label mapping, checkpoint loading, or post-processing. The fastest way to debug it is to treat the pipeline as a sequence of contracts and verify each contract separately.

Start With the Usual Symptoms

"Weird behavior" in object detection usually means one of these:

• boxes appear in the wrong location
• scores are unexpectedly low or unstable
• classes do not match the objects on screen
• inference returns almost no detections
• training metrics improve, but inference looks wrong

Those symptoms point to different failure points. If you debug the entire stack at once, you waste time. Start by isolating where the numbers first stop making sense.

Verify Input Shape and Preprocessing First

A detection model is sensitive to image shape, channel order, and numeric range. Before blaming the model, print the tensor you actually pass into inference.

1import tensorflow as tf
2
3image = tf.random.uniform(shape=(480, 640, 3), maxval=255, dtype=tf.int32)
4image = tf.cast(image, tf.uint8)
5
6input_tensor = tf.expand_dims(tf.cast(image, tf.float32), axis=0)
7print(input_tensor.shape)
8print(input_tensor.dtype)

That simple check catches a surprising number of mistakes. If the model expects a batch dimension and you forget it, the rest of the pipeline becomes misleading. If the preprocessing function expects normalized floats but receives raw integers, scores can look erratic.

Keep the training and inference preprocessing paths aligned. If training resized or normalized images one way and inference does it another way, apparent model instability is often just input mismatch.

Make Sure You Interpret Boxes Correctly

Many models return normalized bounding boxes in the order (ymin, xmin, ymax, xmax). If you treat them as pixel values or swap the coordinate order, the output looks broken even when the detector is correct.

1import numpy as np
2
3box = np.array([0.25, 0.10, 0.75, 0.60])
4height, width = 480, 640
5
6ymin, xmin, ymax, xmax = box
7pixel_box = (
8    int(ymin * height),
9    int(xmin * width),
10    int(ymax * height),
11    int(xmax * width),
12)
13
14print(pixel_box)

If your boxes are shifted, inverted, or drawn outside the image, verify the coordinate convention before changing the model.

Check Label Maps and Class Offsets

Another common source of confusion is label mismatch. The detector may be producing the correct class ID, but your application maps that ID to the wrong label.

Print the raw class IDs and scores before converting them into display names.

1classes = [1, 3, 2]
2scores = [0.98, 0.74, 0.61]
3label_map = {1: "person", 2: "car", 3: "bicycle"}
4
5for class_id, score in zip(classes, scores):
6    print(class_id, label_map.get(class_id, "unknown"), score)

If the dataset label map and the inference-time label map differ, the model can look nonsensical even though the raw predictions are internally consistent.

Separate Training Problems From Inference Problems

If training loss looks reasonable but inference is wrong, the model may be fine and the export or serving path may be broken. Conversely, if inference is noisy on both training and validation examples, the problem is likely earlier.

A practical check is to run inference on one image from the training set that the model should recognize easily. If that still looks wrong, inspect:

• checkpoint loading
• label map alignment
• preprocessing parity
• score threshold and non-max suppression settings

If training truly never converged, then start looking at learning rate, bad annotations, class imbalance, or data pipeline errors.

Use Deterministic Sanity Checks

Randomness makes debugging detection models harder than it needs to be. During investigation, make the setup repeatable.

1import random
2import numpy as np
3import tensorflow as tf
4
5random.seed(7)
6np.random.seed(7)
7tf.random.set_seed(7)

This will not eliminate every source of nondeterminism in every environment, but it reduces noise while you isolate the actual bug.

Common Pitfalls

The most common mistake is debugging rendered boxes instead of the raw tensors. Look at class IDs, scores, and normalized boxes before you touch visualization code.

Another mistake is mixing training and inference preprocessing. A tiny resize or normalization mismatch can look like a model failure.

A third issue is using the wrong label map or an off-by-one class index.

Finally, do not call the behavior "weird" until you have checked the boring explanations first. In detection systems, the boring explanations win most of the time.

Summary

• Debug object detection pipelines one contract at a time: input, model output, label mapping, and rendering.
• Verify tensor shape, dtype, and preprocessing before changing the model.
• Confirm whether boxes are normalized and in (ymin, xmin, ymax, xmax) order.
• Print raw class IDs and scores to validate label-map alignment.
• Compare behavior on an easy training image to separate training issues from inference issues.
• Use deterministic seeds during debugging so the real failure is easier to isolate.