Hadoop
Hive
CSV
Data Loading
Local Machine

Hadoop/Hive Loading data from .csv on a local machine

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design

Introduction

Loading a CSV file into Hive is mostly about understanding where the file lives and how Hive reads it. The phrase "local machine" is the important part, because LOAD DATA LOCAL INPATH uses the file system visible to the Hive client process, not HDFS.

Understand LOCAL Versus HDFS

Hive tables usually store data in HDFS. A CSV on your laptop or on the server where you are running the Hive client is outside HDFS until you load or copy it in.

That gives you two main options:

  • Load directly from the local file system with LOAD DATA LOCAL INPATH
  • Copy the file to HDFS first, then load it with LOAD DATA INPATH

If you are on a single-node test setup, those may look like the same machine. In a real cluster, they are often very different paths and permissions.

Create a Table That Matches the File

Suppose the CSV looks like this:

text
1,Alice,1200
2,Bob,950
3,Carla,1430

You can define a matching Hive table like this:

sql
1CREATE TABLE sales_raw (
2    id INT,
3    customer STRING,
4    amount INT
5)
6ROW FORMAT DELIMITED
7FIELDS TERMINATED BY ','
8STORED AS TEXTFILE;

This simple table works for plain comma-separated rows. If the CSV contains quoted commas or escaped characters, use a CSV serde instead of plain delimiter parsing.

sql
1CREATE TABLE sales_csv (
2    id INT,
3    customer STRING,
4    amount INT
5)
6ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.OpenCSVSerde'
7STORED AS TEXTFILE;

Load the File From the Local Machine

If the file is on the same machine where you run hive or beeline, use:

sql
LOAD DATA LOCAL INPATH '/Users/me/data/sales.csv'
INTO TABLE sales_raw;

LOCAL tells Hive to read from the local file system first and then copy or move the data into the table location. Without LOCAL, Hive assumes the source path already exists in HDFS.

After loading, verify the rows:

sql
SELECT * FROM sales_raw;

This quick check catches delimiter mistakes and schema mismatches early.

It also confirms that the file path and client machine assumptions were correct before you continue with downstream queries. That small verification step saves time later.

Alternative: Put the File in HDFS First

For repeatable pipelines, many teams prefer to stage files in HDFS explicitly:

bash
hdfs dfs -mkdir -p /data/uploads
hdfs dfs -put -f sales.csv /data/uploads/sales.csv

Then load from that HDFS location:

sql
LOAD DATA INPATH '/data/uploads/sales.csv'
INTO TABLE sales_raw;

This makes the ingestion path clearer and is often easier to automate in real environments.

It also separates the "file transfer" step from the "table load" step, which makes failures easier to diagnose.

Common Pitfalls

  • Forgetting LOCAL is the most common mistake when the file is not already in HDFS.
  • A simple delimited table is not a full CSV parser, so quoted commas need a proper serde.
  • Header rows can become bad data unless you remove them or configure the table to skip them.
  • If you run beeline remotely, the file path must exist on the machine running the client command, not just somewhere in the cluster.

Summary

  • Use LOAD DATA LOCAL INPATH when the CSV is on the machine running the Hive client.
  • Use LOAD DATA INPATH when the file is already in HDFS.
  • Make the Hive table match the file format, including delimiter and quoting behavior.
  • Always query a few rows after loading so bad parsing does not go unnoticed.

Related reading
Course
Beginner
27 lessons
10 hours
System Design Fundamentals

Build a strong foundation in designing scalable, reliable distributed systems.

View the course
Track what you have practised

A free account saves your progress, solutions and study plan across every problem on Codemia.

System Design practice on Codemia

Work through 120+ system design problems with detailed solutions, from rate limiters to multi-region storage.

Practice system design

All Rights Reserved.