Python
Download
Web Scraping
Python 3
File Handling

Download file from web in Python 3

Interview Questions practice on Codemia

Over 8,000 real interview questions from top companies, searchable by company and role.

Browse interview questions

Introduction

Downloading a file in Python 3 is easy, but the right approach depends on file size, error handling needs, and whether you want a third-party library. For quick scripts, the standard library may be enough. For real applications, requests with streaming is usually the most practical option because it gives you better control over status checks, chunking, and timeouts.

A Simple Standard Library Approach

Python’s standard library can download a file without extra dependencies.

python
1from urllib.request import urlretrieve
2
3url = "https://example.com/data.csv"
4filename = "data.csv"
5
6urlretrieve(url, filename)
7print("downloaded", filename)

This is convenient for quick scripts, but it offers less control than requests, especially for streaming and advanced error handling.

A Better General-Purpose Pattern With requests

For most real-world code, requests is a stronger default.

python
1import requests
2
3url = "https://example.com/data.csv"
4outfile = "data.csv"
5
6with requests.get(url, stream=True, timeout=30) as response:
7    response.raise_for_status()
8    with open(outfile, "wb") as f:
9        for chunk in response.iter_content(chunk_size=8192):
10            if chunk:
11                f.write(chunk)
12
13print("downloaded", outfile)

This approach is better because it:

  • checks HTTP status explicitly,
  • avoids loading the whole file into memory,
  • works well for large downloads.

Why Streaming Matters

If you call response.content on a very large file, Python may load the whole body into memory at once. Streaming avoids that by processing the response incrementally.

That is usually what you want for:

  • large CSV exports,
  • binary assets,
  • backups,
  • media downloads.

For small files the difference may be negligible, but streaming is a safer default in utility code.

Preserve Or Choose The Output Filename

Sometimes the destination filename is known in advance. Other times you want to derive it from the URL.

python
1from urllib.parse import urlparse
2from pathlib import Path
3
4url = "https://example.com/files/report.pdf"
5filename = Path(urlparse(url).path).name or "download.bin"
6print(filename)

This is useful when building a downloader that saves many files without hardcoding each output name.

Handle Errors Explicitly

Real downloads can fail because of:

  • DNS issues,
  • timeouts,
  • 404 or 500 responses,
  • TLS problems,
  • interrupted connections.

A small defensive wrapper helps.

python
1import requests
2
3
4def download(url, outfile):
5    try:
6        with requests.get(url, stream=True, timeout=30) as response:
7            response.raise_for_status()
8            with open(outfile, "wb") as f:
9                for chunk in response.iter_content(chunk_size=8192):
10                    if chunk:
11                        f.write(chunk)
12    except requests.RequestException as err:
13        print("download failed:", err)
14        raise

That is much better than silently saving partial or invalid files.

Binary Versus Text Files

Download files in binary mode unless you have a strong reason not to.

python
with open("file.bin", "wb") as f:
    ...

Binary mode avoids newline conversion issues and works correctly for text, images, PDFs, archives, and most other web payloads.

If you truly want decoded text, decode it intentionally after download rather than assuming everything is text.

A Good Rule Of Thumb

Use this decision guide:

  • tiny throwaway script: urllib.request.urlretrieve,
  • normal application code: requests with streaming,
  • very large files or resumable transfers: consider more specialized tooling.

That keeps the solution proportional to the problem.

Common Pitfalls

  • Loading large files fully into memory with response.content.
  • Forgetting to call raise_for_status() and treating a 404 page as a successful download.
  • Omitting timeouts and letting network calls hang indefinitely.
  • Writing in text mode when binary mode is the safer default.
  • Assuming a URL always ends with a usable filename.

Summary

  • Python 3 can download files with either the standard library or requests.
  • 'requests with streaming is usually the best general-purpose approach.'
  • Stream large files in chunks instead of loading them entirely into memory.
  • Check status codes and handle network errors explicitly.
  • Save downloads in binary mode unless you intentionally want decoded text processing.

Related reading
Free course
Beginner
7 lessons
2 hours
Tackling System Design Interview Problems

A short course that equips you with the skills to approach system design interviews methodically.

Start the free course
Track what you have practised

A free account saves your progress, solutions and study plan across every problem on Codemia.

Interview Questions practice on Codemia

Over 8,000 real interview questions from top companies, searchable by company and role.

Browse interview questions

All Rights Reserved.