Hadoop/Hive Loading data from .csv on a local machine
System Design practice on Codemia
Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.
Introduction
Loading a CSV file into Hive is mostly about understanding where the file lives and how Hive reads it. The phrase "local machine" is the important part, because LOAD DATA LOCAL INPATH uses the file system visible to the Hive client process, not HDFS.
Understand LOCAL Versus HDFS
Hive tables usually store data in HDFS. A CSV on your laptop or on the server where you are running the Hive client is outside HDFS until you load or copy it in.
That gives you two main options:
- Load directly from the local file system with
LOAD DATA LOCAL INPATH - Copy the file to HDFS first, then load it with
LOAD DATA INPATH
If you are on a single-node test setup, those may look like the same machine. In a real cluster, they are often very different paths and permissions.
Create a Table That Matches the File
Suppose the CSV looks like this:
You can define a matching Hive table like this:
This simple table works for plain comma-separated rows. If the CSV contains quoted commas or escaped characters, use a CSV serde instead of plain delimiter parsing.
Load the File From the Local Machine
If the file is on the same machine where you run hive or beeline, use:
LOCAL tells Hive to read from the local file system first and then copy or move the data into the table location. Without LOCAL, Hive assumes the source path already exists in HDFS.
After loading, verify the rows:
This quick check catches delimiter mistakes and schema mismatches early.
It also confirms that the file path and client machine assumptions were correct before you continue with downstream queries. That small verification step saves time later.
Alternative: Put the File in HDFS First
For repeatable pipelines, many teams prefer to stage files in HDFS explicitly:
Then load from that HDFS location:
This makes the ingestion path clearer and is often easier to automate in real environments.
It also separates the "file transfer" step from the "table load" step, which makes failures easier to diagnose.
Common Pitfalls
- Forgetting
LOCALis the most common mistake when the file is not already in HDFS. - A simple delimited table is not a full CSV parser, so quoted commas need a proper serde.
- Header rows can become bad data unless you remove them or configure the table to skip them.
- If you run
beelineremotely, the file path must exist on the machine running the client command, not just somewhere in the cluster.
Summary
- Use
LOAD DATA LOCAL INPATHwhen the CSV is on the machine running the Hive client. - Use
LOAD DATA INPATHwhen the file is already in HDFS. - Make the Hive table match the file format, including delimiter and quoting behavior.
- Always query a few rows after loading so bad parsing does not go unnoticed.
Related reading
- Handling very large numbers in Python
- hdfs moveFromLocal does not distribute replica blocks across data nodes
- Heuristic for finding elements that appears often together in a big data set
- Hierarchical clustering of 1 million objects
- hive remove stuff from distributed cache
- How can I access S3/S3n from a local Hadoop 2.6 installation?
- How can I control the number of output files written from Spark DataFrame?
- How can I get the most frequent 100 numbers out of 4,000,000,000 numbers?

System Design Fundamentals
Build a strong foundation in designing scalable, reliable distributed systems.
View the courseTrack what you have practised
A free account saves your progress, solutions and study plan across every problem on Codemia.
System Design practice on Codemia
Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.