Introduction

Sometimes a Python dependency is not published on PyPI or you need a specific fix from a branch or commit. In these cases, requirements.txt can install directly from a GitHub repository. The key is using pip VCS URL syntax correctly and pinning to a deterministic reference.

Basic GitHub Dependency Syntax in requirements.txt

The standard format for a GitHub source is:

git+https://github.com/OWNER/REPO.git@REF

Where REF can be a tag, branch, or commit hash.

Examples:

git+https://github.com/pallets/[email protected]
git+https://github.com/psf/requests.git@main
git+https://github.com/your-org/your-lib.git@9f0c8a1

For reproducibility, prefer tags or commit hashes over moving branch names.

Use PEP 508 Direct Reference with Package Name

Modern pip supports explicit package naming with direct references. This is cleaner for tooling.

my-lib @ git+https://github.com/your-org/[email protected]

This style is often easier to read and avoids ambiguity about installed package name.

For extras:

my-lib[cli] @ git+https://github.com/your-org/[email protected]

Install Private GitHub Repositories

For private repositories, HTTPS with token or SSH can be used.

HTTPS token example using environment variable interpolation in CI scripts:

pip install "git+https://${GITHUB_TOKEN}@github.com/your-org/[email protected]"

SSH example in requirements:

git+ssh://[email protected]/your-org/[email protected]

SSH requires deploy keys or agent configuration on the machine where installation runs.

Pinning Strategy and Reproducibility

Avoid unpinned main unless you explicitly want floating updates. Stable environments should pin exact commit hashes.

my-lib @ git+https://github.com/your-org/my-lib.git@9f0c8a1d6d2a8e9c4b4a

Commit pinning ensures repeated installs produce the same dependency code.

If you also generate lock files, treat VCS references as first-class dependencies and keep commit references updated intentionally.

Install from a Repository Subdirectory

Some repositories contain multiple packages and put Python metadata in a subdirectory. You can point pip to that subdirectory using a fragment.

my-lib @ git+https://github.com/your-org/[email protected]#subdirectory=python/my_lib

This tells pip where to find pyproject.toml or setup configuration inside the repository tree.

Editable Installs for Local Development

During active development against a Git source, editable installs can speed iteration.

pip install -e \"git+https://github.com/your-org/my-lib.git@feature-branch#egg=my-lib\"

Editable mode is convenient for local workflows, but production deployments should usually use pinned, non-editable installs for predictability.

Migration Notes for Modern Packaging

If your project uses pyproject.toml, direct references still work, but build backends and metadata must be valid. Test installation in clean virtual environments to catch packaging issues early.

When packaging fails, run pip with verbose mode and check build logs. Most failures come from missing build dependencies, invalid metadata, or incorrect repository layout assumptions.

Organize Requirements for Team Workflows

Many projects split dependency files by environment:

1# requirements/base.txt
2flask==3.0.0
3my-lib @ git+https://github.com/your-org/my-lib.git@9f0c8a1
4
5# requirements/dev.txt
6-r base.txt
7pytest==8.2.0

This keeps core runtime dependencies stable while allowing local dev tooling to vary.

Install with:

pip install -r requirements/dev.txt

Verify What Pip Installed

After install, verify the source and version details.

pip show my-lib
pip freeze | grep my-lib

For deeper checks:

python -c "import my_lib, inspect; print(my_lib.__file__)"

This confirms you are using the intended package location.

Common Pitfalls

A common mistake is pinning to a branch and assuming deterministic builds. Branch heads move, so environments can change unexpectedly. Prefer tags or commits for stable releases.

Another issue is missing package metadata in the target repository. If setup configuration is incomplete, pip install can fail even with correct URL syntax.

Developers also leak personal access tokens by committing tokenized URLs into source control. Use environment variables or secret managers instead of hardcoding credentials.

Summary

• Use pip VCS URL syntax to install directly from GitHub.
• Prefer PEP 508 direct references for clarity.
• Pin tags or commits, not moving branches, for reproducibility.
• Use SSH or CI-managed tokens for private repositories.
• Verify installed source and keep dependency files organized by environment.