Introduction

The TensorFlow Object Detection API is a powerful toolkit developed by the Google Brain team to streamline the implementation of complex object detection models. It supports a variety of architectural choices and pretrained models to cater to diverse use cases. However, some users have encountered a recurring issue where the global steps counter does not show up during training, making it difficult to track training progress effectively. This article delves into this problem, provides technical insights, and suggests solutions to address it.

Understanding Global Steps in TensorFlow

Global steps in TensorFlow, represented by a `tf.Variable`, are used to keep track of the number of batches (or iterations) processed during training. This variable is generally incremented with every training step, serving as a global counter for the entire training process. It is pivotal for tasks like logging progress, decaying learning rates, and managing checkpoints.

In TensorFlow's Object Detection API, the global steps variable should ideally be initialized, incremented, and logged. However, misconfigurations or errors can sometimes lead to its value not being displayed, complicating the monitoring process.

Common Causes for Global Steps Not Displaying

Below are some prevalent causes for the global steps not being displayed when using the TensorFlow Object Detection API:

1. Incorrect Configuration Settings:
   • The configuration files used to train object detection models have parameters that dictate how progress is logged. Neglecting to configure these settings accurately can lead to an absence of global step updates in the output.
2. Logging Level Misconfiguration:
   • TensorFlow has different logging levels, such as `INFO`, `WARNING`, `ERROR`, and `DEBUG`. If the logging level is set too high, certain less critical logs, including those for global steps, may not appear.
3. Impediments in TensorFlow Estimator:
   • The `tf.estimator.Estimator` API is utilized under the hood by the Object Detection API for training. Improper usage or errors within custom inputs could prevent global steps from being displayed.
4. Incorrect Event File Location:
   • TensorBoard, often used for monitoring TensorFlow training, may not be pointed to the correct event files generated during training. Consequently, global steps and other important metrics might not render properly in the UI.

Technical Solution to Display Global Steps

The technical solution to ensure that the global steps are displayed involves inspecting and adjusting several configurations and code sections. Here's a step-by-step guide:

Configuration Checks

1. Inspect pipeline configuration: Ensure that the `fine_tune_checkpoint` and `from_detection_checkpoint` settings within the pipeline configuration are correctly defined. A mismatch here can disrupt the flow of training logs.
2. Adjust logging settings: In the Python script used to start the training process, adjust the logging level to `INFO` globally as follows: