Introduction

pip install -r requirements.txt fails fast by design, which is correct for CI and production reproducibility. The challenge appears in local developer setup where one optional package can block all progress. The practical solution is to separate mandatory dependencies from optional tooling and make failures visible instead of silently ignored.

Why pip Stops on First Failure

pip returns non-zero on installation failure so automation can fail reliably. This protects environments from partial dependency states.

In local development, strict behavior may be inconvenient for optional packages with platform-specific build requirements.

Examples:

• Graph visualization libraries requiring native headers.
• GPU packages on non-GPU machines.
• OS-specific wheel gaps.

Split Core and Optional Requirements

Use separate files for mandatory and optional dependencies.

# requirements.txt
-r requirements-core.txt
-r requirements-optional.txt

1# requirements-core.txt
2fastapi==0.116.0
3uvicorn==0.35.0
4sqlalchemy==2.0.36

# requirements-optional.txt
pygraphviz==1.14
matplotlib==3.9.2

Install core strictly first:

python -m pip install -r requirements-core.txt

If core fails, stop and fix.

Best-Effort Optional Install with Failure Report

For optional packages, install per line and collect failures.

1#!/usr/bin/env bash
2set -u
3
4failed=()
5while IFS= read -r pkg; do
6  [[ -z "$pkg" || "$pkg" == \#* ]] && continue
7  echo "Installing optional package: $pkg"
8  if ! python -m pip install "$pkg"; then
9    failed+=("$pkg")
10  fi
11done < requirements-optional.txt
12
13if [[ ${#failed[@]} -gt 0 ]]; then
14  printf '%s\n' "Optional install failures:" "${failed[@]}"
15  printf '%s\n' "${failed[@]}" > optional-install-failures.txt
16fi

This keeps setup moving while preserving transparency.

Use Environment Markers to Avoid Unneeded Attempts

For platform-specific packages, use markers directly in requirement lines.

pywin32==306; platform_system == "Windows"
uvloop==0.20.0; platform_system != "Windows"

Markers reduce predictable failures and simplify onboarding.

Keep Reproducibility with Constraints

For larger projects, pair requirements with constraints.

python -m pip install -r requirements-core.txt -c constraints.txt

Constraints keep versions stable across machines and CI runs.

Diagnose Optional Failures Properly

Do not ignore optional failures forever. Capture reason and classify:

• Missing system packages.
• Unsupported Python version.
• Broken wheel for current platform.

Useful commands:

python -m pip install -v pygraphviz==1.14
python -m pip debug --verbose
python -m pip index versions pygraphviz

Document system prerequisites near requirements files.

CI and Production Policy

In CI and production images, keep strict all-or-nothing behavior for required dependencies. Optional tolerance should be a local developer convenience, not deployment policy.

Recommended policy:

• CI installs core requirements strictly.
• Optional installs run in separate non-blocking job if needed.
• Failures are logged and tracked.

Team Workflow Guidance

Maintain clear ownership of optional dependency groups. If one optional package fails repeatedly, either fix platform support or remove it from shared setup instructions. Unowned optional packages become recurring friction points.

Prefer Optional Extras for New Projects

If you control packaging, dependency extras often provide a cleaner model than one monolithic requirements file. Users can opt into feature groups explicitly.

[project.optional-dependencies]
viz = ["pygraphviz==1.14", "matplotlib==3.9.2"]
dev = ["pytest==8.3.3", "ruff==0.7.3"]

Install only what is needed:

python -m pip install .[dev]

This keeps core environments stable and avoids repeated optional-install failures.

Common Pitfalls

• Treating all dependencies as equally mandatory.
• Continuing after optional failures without generating any report.
• Applying tolerant install behavior in production pipelines.
• Not documenting why a dependency is optional.
• Running installs outside virtual environments and confusing diagnostics.

Summary

• pip fail-fast behavior is correct for deterministic environments.
• Split mandatory and optional dependencies explicitly.
• Keep core installs strict and optional installs best-effort with reporting.
• Use environment markers and constraints to reduce avoidable failures.
• Keep CI and production dependency policy strict and reproducible.