Introduction

In old Hadoop MapReduce APIs, DistributedCache.getCacheFiles() and DistributedCache.getLocalCacheFiles() sound similar but answer different questions. One tells you which cache files were configured for the job as distributed URIs, while the other tells you where those files were localized on the task node's local filesystem.

getCacheFiles() Returns the Declared Cache URIs

When a job adds files to the distributed cache, Hadoop stores their source locations as URIs.

1import java.net.URI;
2import org.apache.hadoop.filecache.DistributedCache;
3import org.apache.hadoop.mapred.JobConf;
4
5JobConf conf = new JobConf();
6DistributedCache.addCacheFile(new URI("hdfs:///data/lookup/countries.csv"), conf);
7
8URI[] uris = DistributedCache.getCacheFiles(conf);
9for (URI uri : uris) {
10    System.out.println(uri);
11}

The important point is that these are not the paths you normally open inside a mapper task. They are the original configured resource locations, often on HDFS or another supported filesystem.

Use this when you want to inspect job configuration or verify that the correct files were attached.

getLocalCacheFiles() Returns the Localized Paths

Before a task runs, Hadoop copies distributed-cache resources to the local node. getLocalCacheFiles() returns those local paths.

1import java.io.BufferedReader;
2import java.io.FileReader;
3import org.apache.hadoop.fs.Path;
4import org.apache.hadoop.filecache.DistributedCache;
5import org.apache.hadoop.mapred.JobConf;
6
7JobConf conf = new JobConf();
8Path[] localPaths = DistributedCache.getLocalCacheFiles(conf);
9
10for (Path p : localPaths) {
11    System.out.println("localized: " + p);
12}

These paths are what task code typically opens when it needs to read reference data efficiently from local disk rather than from HDFS on every record.

Example mapper setup:

1public void configure(JobConf conf) {
2    try {
3        Path[] localFiles = DistributedCache.getLocalCacheFiles(conf);
4        if (localFiles != null) {
5            try (BufferedReader reader = new BufferedReader(new FileReader(localFiles[0].toString()))) {
6                System.out.println(reader.readLine());
7            }
8        }
9    } catch (Exception e) {
10        throw new RuntimeException(e);
11    }
12}

Think of the Difference as Remote Versus Local View

A practical mental model is:

• 'getCacheFiles() = what was declared for the job'
• 'getLocalCacheFiles() = where it ended up on this node'

That distinction matters because a file can originate at hdfs:///... but be accessed locally as a task-specific cached copy under a different path.

If your mapper needs to open the actual file during execution, the local path is usually what you want.

Remember the API Age

One more wrinkle is that DistributedCache itself belongs to older Hadoop APIs and has been deprecated in favor of newer configuration patterns such as Job.addCacheFile(...). The conceptual distinction still matters, though, because localization behavior did not disappear.

Newer-style job setup:

1import java.net.URI;
2import org.apache.hadoop.mapreduce.Job;
3
4Job job = Job.getInstance();
5job.addCacheFile(new URI("hdfs:///data/lookup/countries.csv"));

The same underlying idea remains: configured cache resource versus localized on-node file.

When Each Method Is Useful

Use getCacheFiles() when:

• validating job setup
• logging configured resources
• comparing declared cache URIs

Use getLocalCacheFiles() when:

• opening the cached file in mapper or reducer code
• checking whether localization succeeded
• reading local task-node copies for speed

Keeping those roles separate makes the API much less confusing.

Common Pitfalls

• Trying to open a URI from getCacheFiles() as if it were already a local file path.
• Assuming both methods return the same information in a different type.
• Forgetting that distributed-cache resources are localized before task execution.
• Using old DistributedCache examples without noticing API deprecation in newer Hadoop versions.
• Debugging the wrong layer by checking local paths when the real problem is that the job never added the URI correctly.

Summary

• 'getCacheFiles() returns the original configured cache URIs.'
• 'getLocalCacheFiles() returns the localized paths on the task node.'
• Use the local form when task code needs to read the cached file.
• Use the URI form when inspecting job configuration.
• Even in newer Hadoop APIs, the remote-versus-local distinction still matters.