Python
requirements.txt
package management
dependency management
alternative packages

requirements.txt - How to mark alternative packages

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Introduction

Python's requirements.txt does not natively support "either/or" alternative packages. Each line specifies one package that must be installed. To handle alternatives (e.g., ujson or json, psycopg2 or psycopg2-binary), use extras in setup.py/pyproject.toml, environment markers, multiple requirements files, or conditional installation in a script. The standard approach for modern Python projects is to define optional dependency groups in pyproject.toml and let users choose.

Basic requirements.txt Syntax

text
1# Pin exact version
2requests==2.31.0
3
4# Version range
5flask>=2.0,<3.0
6
7# Minimum version
8numpy>=1.24
9
10# Any version
11beautifulsoup4
12
13# From a URL
14git+https://github.com/user/repo.git@main#egg=package
15
16# Editable local install
17-e ./my-local-package

Each line specifies one package. There is no syntax for "install A or B".

Environment Markers (Platform-Specific Alternatives)

text
1# Install different packages based on platform
2pywin32>=306; sys_platform == "win32"
3pyobjc>=9.0; sys_platform == "darwin"
4
5# Install based on Python version
6dataclasses>=0.6; python_version < "3.7"
7typing_extensions>=4.0; python_version < "3.10"
8
9# Install based on implementation
10cffi>=1.15; implementation_name == "cpython"

Environment markers (PEP 508) conditionally install packages based on platform, Python version, or implementation. They handle platform-specific alternatives but not user-choice alternatives.

Multiple Requirements Files

text
1# requirements-base.txt
2flask>=2.0
3sqlalchemy>=2.0
4celery>=5.3
5
6# requirements-postgres.txt
7-r requirements-base.txt
8psycopg2-binary>=2.9
9
10# requirements-mysql.txt
11-r requirements-base.txt
12mysqlclient>=2.1
13
14# requirements-dev.txt
15-r requirements-base.txt
16pytest>=7.0
17black>=23.0
bash
1# User chooses which to install
2pip install -r requirements-postgres.txt
3# or
4pip install -r requirements-mysql.txt

This is the simplest way to offer alternatives. -r includes another file, so shared dependencies are not duplicated.

Optional Dependencies in pyproject.toml

toml
1# pyproject.toml
2[project]
3name = "myapp"
4version = "1.0.0"
5dependencies = [
6    "flask>=2.0",
7    "sqlalchemy>=2.0",
8]
9
10[project.optional-dependencies]
11postgres = ["psycopg2-binary>=2.9"]
12mysql = ["mysqlclient>=2.1"]
13redis = ["redis>=5.0"]
14fast = ["ujson>=5.0", "uvloop>=0.19; sys_platform != 'win32'"]
15dev = ["pytest>=7.0", "black>=23.0", "mypy>=1.0"]
16all = ["psycopg2-binary>=2.9", "redis>=5.0", "ujson>=5.0"]
bash
1# Install with specific extras
2pip install myapp[postgres]
3pip install myapp[postgres,redis]
4pip install myapp[all]
5pip install -e ".[dev,postgres]"

This is the modern, recommended approach. Users choose which optional feature sets to install using the [extra] syntax.

Extras in setup.py (Legacy)

python
1# setup.py
2from setuptools import setup
3
4setup(
5    name="myapp",
6    install_requires=[
7        "flask>=2.0",
8        "sqlalchemy>=2.0",
9    ],
10    extras_require={
11        "postgres": ["psycopg2-binary>=2.9"],
12        "mysql": ["mysqlclient>=2.1"],
13        "fast-json": ["ujson>=5.0"],
14        "dev": ["pytest>=7.0"],
15    },
16)
bash
pip install myapp[postgres]
pip install myapp[postgres,fast-json]

Conditional Installation Script

python
1#!/usr/bin/env python3
2"""install.py — install with fallback packages"""
3import subprocess
4import sys
5
6def install(package, fallback=None):
7    try:
8        subprocess.check_call([sys.executable, "-m", "pip", "install", package])
9        return True
10    except subprocess.CalledProcessError:
11        if fallback:
12            print(f"Failed to install {package}, trying {fallback}")
13            subprocess.check_call([sys.executable, "-m", "pip", "install", fallback])
14            return True
15        return False
16
17# Try optimized packages first, fall back to pure Python
18install("ujson", fallback="json")
19install("psycopg2", fallback="psycopg2-binary")
20install("uvloop; sys_platform != 'win32'", fallback=None)

This approach tries to install the preferred package and falls back to an alternative on failure. Useful for C-extension packages that may not compile on all systems.

Using pip Constraints Files

text
1# constraints.txt — pin versions without requiring installation
2psycopg2-binary==2.9.9
3redis==5.0.1
4ujson==5.9.0
bash
# Install requirements with version constraints
pip install -r requirements.txt -c constraints.txt

Constraints files pin versions for packages that appear in requirements.txt but do not force their installation. They are useful for locking versions in CI/CD without changing the requirements file.

Common Pitfalls

  • Expecting requirements.txt to support OR syntax: There is no A | B or A or B syntax in requirements.txt. Each line installs exactly one package. Use multiple requirements files, pyproject.toml extras, or an install script for alternatives.
  • Using psycopg2 in requirements when psycopg2-binary is easier: psycopg2 requires libpq-dev and a C compiler to build from source. psycopg2-binary includes pre-built binaries. For development and CI, use -binary; for production, compile from source. Offer both as alternatives.
  • Forgetting environment markers syntax: Markers use == for comparison, not =. Valid markers include sys_platform, python_version, implementation_name, and os_name. An invalid marker silently does nothing.
  • Not using -r to include base files: Duplicating shared dependencies across requirements-postgres.txt and requirements-mysql.txt leads to version drift. Use -r requirements-base.txt to include shared dependencies from one place.
  • Mixing pyproject.toml and setup.py for extras: Having optional dependencies defined in both pyproject.toml and setup.py causes confusion about which takes precedence. Use pyproject.toml for new projects; setup.py only for backward compatibility.

Summary

  • requirements.txt does not support alternative/OR package syntax — each line installs one package
  • Use multiple requirements files (-r base.txt) to offer database, cache, or feature alternatives
  • Use [project.optional-dependencies] in pyproject.toml for the modern approach with pip install myapp[extra]
  • Use environment markers (; sys_platform == "win32") for platform-specific packages
  • Use a conditional install script for try-first/fallback patterns (e.g., C extension vs pure Python)
  • Constraints files (-c constraints.txt) pin versions without requiring installation

Course illustration
Course illustration

All Rights Reserved.