TensorBoard
502 Bad Gateway
error troubleshooting
machine learning
server issues

'View TensorBoard' goes to 502 Bad Gateway

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Introduction

A 502 Bad Gateway error usually means your browser reached a proxy, but that proxy could not get a valid response from TensorBoard itself. In other words, the problem is often not "TensorBoard is broken" but "the network path between the proxy and TensorBoard is wrong, slow, or dead."

What a 502 Means in This Context

TensorBoard often runs behind another layer:

  • Jupyter or a hosted notebook environment
  • Nginx or Apache as a reverse proxy
  • Kubernetes ingress
  • SSH port forwarding through a remote machine

In all of those setups, your browser does not usually talk to TensorBoard directly. The proxy tries to reach the TensorBoard process on a local port, and if that upstream connection fails, you see 502.

First Check That TensorBoard Is Actually Running

Before touching proxy configuration, confirm the process is alive and listening on the port you expect.

bash
tensorboard --logdir ./runs --host 127.0.0.1 --port 6006

In another terminal, test it locally on the same machine:

bash
curl -I http://127.0.0.1:6006/

If that request fails, the issue is inside the TensorBoard process or its startup command. Common causes include:

  • Wrong --logdir
  • Port already in use
  • TensorBoard crashed on startup
  • The process is bound to a different interface or port

If the local curl works but the browser still gets 502, the proxy layer is the problem.

Common Upstream Mismatches

One frequent issue is binding TensorBoard to 127.0.0.1 while the proxy expects a service reachable on another interface or container IP. In containerized or remote environments, binding to all interfaces is often necessary:

bash
tensorboard --logdir /data/logs --host 0.0.0.0 --port 6006

Another issue is a plain port mismatch. For example, TensorBoard may be listening on 6007, while the proxy still forwards to 6006.

You should also confirm that the target path is correct. Some environments proxy TensorBoard under a subpath such as /tensorboard/ instead of /. If the proxy does not preserve headers or path prefixes correctly, the upstream request can fail.

Reverse Proxy Example

Here is a minimal Nginx configuration for forwarding requests to a local TensorBoard instance:

nginx
1server {
2    listen 80;
3    server_name example.com;
4
5    location /tensorboard/ {
6        proxy_pass http://127.0.0.1:6006/;
7        proxy_http_version 1.1;
8        proxy_set_header Host $host;
9        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
10        proxy_set_header X-Forwarded-Proto $scheme;
11        proxy_read_timeout 300;
12    }
13}

If this proxy points to the wrong host or wrong port, 502 is the expected result. If the timeout is too short and the upstream takes too long to respond, some gateways also return 502 or 504 depending on the component involved.

Hosted Notebook and Kubernetes Cases

In notebook platforms, the "View TensorBoard" button often assumes a specific port and internal route. If you launched TensorBoard manually on another port, the UI link may target the wrong upstream.

In Kubernetes, the usual chain is:

  • TensorBoard container
  • Service
  • Ingress or gateway

Check each hop independently:

bash
1kubectl get pods
2kubectl get svc
3kubectl get ingress
4kubectl port-forward svc/tensorboard 6006:6006

If port-forwarding works but ingress returns 502, the problem is ingress configuration, not TensorBoard.

Logs Are Usually the Fastest Answer

A 502 page in the browser tells you almost nothing by itself. The useful information is in logs:

  • TensorBoard startup logs
  • Reverse-proxy error logs
  • Container or pod logs
  • Notebook server logs if a managed environment launches the proxy

For example, Nginx may say it could not connect to the upstream, while TensorBoard logs may show it never started because the log directory path was invalid.

A Simple Troubleshooting Sequence

Use this order so you do not change too many layers at once:

  1. Start TensorBoard manually and confirm the port.
  2. curl the TensorBoard URL locally on the host that runs it.
  3. Test the service from the proxy host or container.
  4. Inspect proxy logs for upstream connection errors.
  5. Only then adjust ingress, rewrite, or timeout settings.

That sequence isolates whether the failure is process-level, host-level, or proxy-level.

Common Pitfalls

The most common pitfall is assuming the browser-facing URL and the TensorBoard bind address are the same thing. They are often separated by one or more proxy layers.

Another mistake is binding TensorBoard to localhost inside a container, then expecting another container or ingress to reach it. 127.0.0.1 is local to that container, not the cluster.

Developers also overlook stale links in hosted notebook environments. The "View TensorBoard" shortcut may point to a previous session, different port, or dead backend.

Finally, do not debug 502 from the browser alone. Without proxy and process logs, you are guessing.

Summary

  • A TensorBoard 502 usually means a proxy cannot reach the TensorBoard upstream.
  • First verify that TensorBoard is running and listening on the expected host and port.
  • Check bind address, port mappings, path prefixes, and proxy timeout settings.
  • In Kubernetes or notebook setups, test each network hop independently.
  • Process logs and proxy logs are the fastest way to identify the real failure.

Course illustration
Course illustration

All Rights Reserved.