Download file from web in Python 3
Interview Questions practice on Codemia
Over 8,000 real interview questions from top companies, searchable by company and role.
Introduction
Downloading a file in Python 3 is easy, but the right approach depends on file size, error handling needs, and whether you want a third-party library. For quick scripts, the standard library may be enough. For real applications, requests with streaming is usually the most practical option because it gives you better control over status checks, chunking, and timeouts.
A Simple Standard Library Approach
Python’s standard library can download a file without extra dependencies.
This is convenient for quick scripts, but it offers less control than requests, especially for streaming and advanced error handling.
A Better General-Purpose Pattern With requests
For most real-world code, requests is a stronger default.
This approach is better because it:
- checks HTTP status explicitly,
- avoids loading the whole file into memory,
- works well for large downloads.
Why Streaming Matters
If you call response.content on a very large file, Python may load the whole body into memory at once. Streaming avoids that by processing the response incrementally.
That is usually what you want for:
- large CSV exports,
- binary assets,
- backups,
- media downloads.
For small files the difference may be negligible, but streaming is a safer default in utility code.
Preserve Or Choose The Output Filename
Sometimes the destination filename is known in advance. Other times you want to derive it from the URL.
This is useful when building a downloader that saves many files without hardcoding each output name.
Handle Errors Explicitly
Real downloads can fail because of:
- DNS issues,
- timeouts,
- 404 or 500 responses,
- TLS problems,
- interrupted connections.
A small defensive wrapper helps.
That is much better than silently saving partial or invalid files.
Binary Versus Text Files
Download files in binary mode unless you have a strong reason not to.
Binary mode avoids newline conversion issues and works correctly for text, images, PDFs, archives, and most other web payloads.
If you truly want decoded text, decode it intentionally after download rather than assuming everything is text.
A Good Rule Of Thumb
Use this decision guide:
- tiny throwaway script:
urllib.request.urlretrieve, - normal application code:
requestswith streaming, - very large files or resumable transfers: consider more specialized tooling.
That keeps the solution proportional to the problem.
Common Pitfalls
- Loading large files fully into memory with
response.content. - Forgetting to call
raise_for_status()and treating a 404 page as a successful download. - Omitting timeouts and letting network calls hang indefinitely.
- Writing in text mode when binary mode is the safer default.
- Assuming a URL always ends with a usable filename.
Summary
- Python 3 can download files with either the standard library or
requests. - '
requestswith streaming is usually the best general-purpose approach.' - Stream large files in chunks instead of loading them entirely into memory.
- Check status codes and handle network errors explicitly.
- Save downloads in binary mode unless you intentionally want decoded text processing.
Related reading
- Download large file in python with requests
- Downloading a picture via urllib and python
- Drop all duplicate rows across multiple columns in Python Pandas
- Drop columns whose name contains a specific string from pandas DataFrame
- Drop data frame columns by name
- Dropping infinite values from dataframes in pandas?
- Dump a NumPy array into a csv file
- Dump a NumPy array into a csv file
.png&w=3840&q=75)
Tackling System Design Interview Problems
A short course that equips you with the skills to approach system design interviews methodically.
Start the free courseTrack what you have practised
A free account saves your progress, solutions and study plan across every problem on Codemia.
Interview Questions practice on Codemia
Over 8,000 real interview questions from top companies, searchable by company and role.