Introduction

Moving data from SQL into pandas is one of the most common steps in a Python workflow. The goal is not just to fetch rows, but to bring them into a DataFrame with correct column names, safe query parameters, and useful types so analysis can start immediately.

Query Directly Into a DataFrame

The most direct approach is to let pandas run the SQL and build the table structure for you. pandas.read_sql_query is usually the best choice because it returns labeled columns and removes the need for manual cursor loops.

1import sqlite3
2import pandas as pd
3
4conn = sqlite3.connect(":memory:")
5cursor = conn.cursor()
6
7cursor.execute("""
8CREATE TABLE orders (
9    order_id INTEGER PRIMARY KEY,
10    customer TEXT NOT NULL,
11    total REAL NOT NULL,
12    created_at TEXT NOT NULL
13)
14""")
15
16cursor.executemany(
17    "INSERT INTO orders (customer, total, created_at) VALUES (?, ?, ?)",
18    [
19        ("Ava", 120.50, "2026-03-01"),
20        ("Liam", 89.99, "2026-03-02"),
21        ("Ava", 42.00, "2026-03-03"),
22    ],
23)
24conn.commit()
25
26query = """
27SELECT customer, total, created_at
28FROM orders
29WHERE total >= ?
30ORDER BY created_at
31"""
32
33df = pd.read_sql_query(query, conn, params=(50,))
34print(df)
35print(df.dtypes)

That produces a DataFrame with named columns instead of a plain list of tuples. In practice, this is easier to inspect, filter, group, and export.

Control Types During Import

A common surprise is that SQL data arrives with types that are technically valid but inconvenient for analysis. Text dates are the usual example. You can fix that while importing instead of converting everything later.

1df = pd.read_sql_query(
2    """
3    SELECT order_id, customer, total, created_at
4    FROM orders
5    """,
6    conn,
7    parse_dates=["created_at"],
8    index_col="order_id",
9)
10
11print(df)
12print(df.index.name)
13print(df.dtypes)

Parsing dates early makes time-based filtering and grouping much simpler. Setting an index during import also avoids an extra cleanup step.

Connect Through SQLAlchemy

For small scripts, a direct database connection is fine. For production applications, SQLAlchemy is often more practical because it standardizes connection handling across multiple database engines.

1from sqlalchemy import create_engine
2import pandas as pd
3
4engine = create_engine("postgresql+psycopg2://app_user:[email protected]:5432/app")
5
6df = pd.read_sql_query(
7    """
8    SELECT id, email, signup_date
9    FROM users
10    WHERE signup_date >= %(cutoff)s
11    """,
12    engine,
13    params={"cutoff": "2026-01-01"},
14)

The pandas side barely changes, which is why this pattern scales well as a project grows from local experiments to a real service.

Convert Existing Cursor Results

Sometimes the query has already been executed and you only have the cursor. In that case, build the DataFrame with the rows plus the column names from cursor.description.

1cursor.execute("SELECT order_id, customer, total FROM orders")
2rows = cursor.fetchall()
3columns = [col[0] for col in cursor.description]
4
5df = pd.DataFrame(rows, columns=columns)
6print(df)

This is perfectly valid, but it is usually a fallback. If you control the query, read_sql_query is shorter and less error-prone.

Work With Large Queries Safely

A query that returns a few thousand rows in development may return millions in production. If the full result is too large, read it in chunks instead of materializing the whole table at once.

1chunk_iter = pd.read_sql_query(
2    "SELECT order_id, customer, total FROM orders",
3    conn,
4    chunksize=1000,
5)
6
7for chunk in chunk_iter:
8    print("rows in chunk:", len(chunk))

Chunked reads are useful for ETL pipelines, reporting jobs, and one-pass transformations where you do not need the entire dataset in memory.

Common Pitfalls

The first mistake is building SQL by concatenating strings. Always pass parameters through the database driver or pandas call, otherwise quoting bugs and SQL injection become real risks.

Another common issue is ignoring data types after import. Check df.dtypes, especially for dates, booleans, decimals, and nullable integers. Incorrect types usually do not fail immediately, but they do cause subtle bugs in filtering and aggregation.

Large queries are another trap. Developers often test on small databases and then hit memory problems later. Select only the columns you need, filter in SQL first, and use chunksize when the result is large.

Finally, do not forget connection management. A short script can open a connection and exit, but services and scheduled jobs should close connections predictably or use an engine that handles pooling.

Summary

• Use pandas.read_sql_query when you want SQL results as a DataFrame.
• Prefer parameters over string formatting in the SQL statement.
• Parse dates and set an index during import when those choices help analysis.
• Use SQLAlchemy when you need a cleaner path across different database backends.
• Read large result sets in chunks instead of loading everything into memory.