Introduction

In TensorFlow’s older distributed runtime model, the master and worker services are the pieces that coordinate and execute a graph across machines. The master is the coordinator that plans and schedules work, while workers are the processes that actually run graph operations on CPUs, GPUs, or other devices.

The distributed TensorFlow mental model

Classic distributed TensorFlow is based on jobs and tasks. A job is a named group of processes such as worker or ps, and each running process inside that job is a task.

The high-level roles are:

• client: builds the graph and starts execution
• master: coordinates graph execution and device placement
• worker: executes ops on local devices

In older code, you often interacted with this through a ClusterSpec and one or more tf.train.Server processes.

What the master service does

The master service is the orchestration layer. Its main responsibilities include:

• receiving session execution requests
• partitioning the graph across available devices
• scheduling graph fragments on workers
• coordinating error propagation and completion

It does not do all the actual numeric work itself. Instead, it decides where pieces of the graph should run and then asks workers to execute them.

What the worker service does

Workers are the processes that own devices and execute graph operations. A worker may run ops on its CPU, GPU, or both, depending on how the graph is partitioned and where the runtime places each node.

If the graph says an op belongs on a GPU hosted by task worker:1, that worker receives the request and runs the op locally. In distributed training, this is where the real matrix multiplication, gradient calculation, and parameter updates occur.

A minimal ClusterSpec example

The code below shows the old-style structure:

1import tensorflow as tf
2
3cluster = tf.train.ClusterSpec({
4    "worker": ["localhost:2222", "localhost:2223"],
5    "ps": ["localhost:2224"],
6})
7
8server = tf.train.Server(cluster, job_name="worker", task_index=0)
9print(server.target)

This does not by itself train a model, but it shows how tasks are grouped into jobs. Once a client session connects to server.target, the runtime can coordinate execution through the master and workers defined by that cluster.

Why parameter servers used to matter

In the same architecture, parameter servers often held model variables while workers computed gradients. That let several workers update shared parameters without each worker owning a complete independent copy of all state.

Modern TensorFlow often prefers higher-level distribution strategies instead of exposing these roles directly, but the older master-worker model explains a lot of historical TensorFlow examples.

What matters in modern TensorFlow

In current TensorFlow code, many developers never touch the master and worker services directly because tf.distribute hides most of the runtime machinery. Even so, the old terminology still appears in legacy jobs, internal tooling, and older articles, so it is useful to know that the master is the coordinator and the workers are the executors.

That distinction helps when reading logs or debugging old distributed setups.

Common Pitfalls

• Thinking the master performs all computation instead of mainly coordinating it.
• Confusing workers with parameter servers in older distributed training examples.
• Reading legacy tf.train.Server documentation as if it were the preferred style for new TensorFlow projects.
• Ignoring job and task numbering, which is essential for distributed process identity.

Summary

• In classic TensorFlow, the master coordinates distributed execution and workers run the actual ops.
• Jobs and tasks identify the processes participating in the cluster.
• Older distributed training often combined workers with parameter servers.
• Modern TensorFlow usually hides these details behind higher-level distribution APIs, but the concepts still matter in legacy systems.