How should I structure a Python package that contains Cython code

Python

Cython

Package Structure

Software Development

Programming Techniques

How should I structure a Python package that contains Cython code

Data Structures & Algorithms practice on Codemia

Step through 300+ coding interview problems with animated visualisers that show the data structure changing as the code runs. The Blind 75 are free.

Practice algorithms

Python

Introduction

A Python package with Cython code keeps .pyx (Cython source) files alongside .py files in the normal package directory, with a setup.py or pyproject.toml that compiles them into C extensions during installation. The key principle: ship both .pyx sources and a pure-Python fallback so users can install with or without a C compiler. The compiled extensions provide speed; the fallback ensures the package always installs.

Recommended Directory Structure

1mypackage/
2├── pyproject.toml
3├── setup.py
4├── README.md
5├── mypackage/
6│   ├── __init__.py
7│   ├── core.py              # Pure Python (fallback)
8│   ├── _core_cy.pyx         # Cython implementation
9│   ├── _core_cy.pxd         # Cython declarations (optional)
10│   ├── utils.py             # Pure Python module
11│   ├── _fast_math.pyx       # Another Cython module
12│   └── _fast_math.pxd
13└── tests/
14    ├── test_core.py
15    └── test_fast_math.py

Convention: prefix Cython modules with _ to indicate they are internal implementations.

pyproject.toml (Modern Build)

toml

1[build-system]
2requires = ["setuptools>=68.0", "wheel", "Cython>=3.0"]
3build-backend = "setuptools.build_meta"
4
5[project]
6name = "mypackage"
7version = "1.0.0"
8requires-python = ">=3.8"

setup.py (Compilation Configuration)

python

1from setuptools import setup, Extension, find_packages
2import os
3
4# Try to import Cython; fall back to pre-generated C files
5try:
6    from Cython.Build import cythonize
7    USE_CYTHON = True
8except ImportError:
9    USE_CYTHON = False
10
11ext = '.pyx' if USE_CYTHON else '.c'
12
13extensions = [
14    Extension(
15        "mypackage._core_cy",
16        sources=[f"mypackage/_core_cy{ext}"],
17    ),
18    Extension(
19        "mypackage._fast_math",
20        sources=[f"mypackage/_fast_math{ext}"],
21    ),
22]
23
24if USE_CYTHON:
25    extensions = cythonize(
26        extensions,
27        compiler_directives={
28            'language_level': '3',
29            'boundscheck': False,
30            'wraparound': False,
31        },
32    )
33
34setup(
35    name="mypackage",
36    packages=find_packages(),
37    ext_modules=extensions,
38)

The Cython Source (.pyx)

cython

1# mypackage/_core_cy.pyx
2# cython: language_level=3
3
4import numpy as np
5cimport numpy as cnp
6
7def fast_sum(cnp.ndarray[double, ndim=1] arr):
8    """Compute sum of array elements using typed memoryview."""
9    cdef double total = 0.0
10    cdef Py_ssize_t i
11    cdef Py_ssize_t n = arr.shape[0]
12
13    for i in range(n):
14        total += arr[i]
15
16    return total

Pure-Python Fallback

python

1# mypackage/core.py
2import numpy as np
3
4def fast_sum(arr):
5    """Pure Python fallback — works without Cython compilation."""
6    return float(np.sum(arr))

init.py with Fallback Import

python

1# mypackage/__init__.py
2try:
3    from mypackage._core_cy import fast_sum
4except ImportError:
5    # Cython extension not compiled — use pure Python
6    from mypackage.core import fast_sum
7
8try:
9    from mypackage._fast_math import matrix_multiply
10except ImportError:
11    from mypackage.math_fallback import matrix_multiply
12
13__all__ = ['fast_sum', 'matrix_multiply']

Users always import from mypackage and get the fastest available implementation transparently.

Building and Installing

bash

1# Development install (compiles Cython in-place)
2pip install -e .
3
4# Build wheel (compiled extension included)
5pip wheel . --no-deps
6
7# Build source distribution (includes .pyx files)
8python -m build --sdist
9
10# Compile Cython files manually (for development)
11python setup.py build_ext --inplace

Including Pre-Generated C Files

Ship .c files alongside .pyx so users without Cython can still compile:

python

1# In setup.py, already handled by the ext = '.pyx' if USE_CYTHON else '.c' pattern
2
3# Generate C files before releasing:
4# cython mypackage/_core_cy.pyx
5# cython mypackage/_fast_math.pyx

toml

1# In MANIFEST.in, include both .pyx and .c
2include mypackage/*.pyx
3include mypackage/*.pxd
4include mypackage/*.c

Cython Declaration Files (.pxd)

cython

1# mypackage/_core_cy.pxd
2# Share type declarations between Cython modules
3
4cdef class Vector:
5    cdef double x, y, z
6    cdef double magnitude(self)

.pxd files are like C header files — they declare types and functions that other .pyx files can cimport.

Testing Both Implementations

python

1# tests/test_core.py
2import pytest
3import numpy as np
4
5class TestFastSum:
6    def test_basic(self):
7        from mypackage import fast_sum
8        arr = np.array([1.0, 2.0, 3.0])
9        assert fast_sum(arr) == pytest.approx(6.0)
10
11    def test_cython_available(self):
12        """Verify Cython extension is compiled."""
13        try:
14            from mypackage._core_cy import fast_sum
15            assert True  # Cython version loaded
16        except ImportError:
17            pytest.skip("Cython extension not compiled")
18
19    def test_fallback(self):
20        """Test pure Python fallback directly."""
21        from mypackage.core import fast_sum
22        arr = np.array([1.0, 2.0, 3.0])
23        assert fast_sum(arr) == pytest.approx(6.0)

CI/CD Configuration

yaml

1# GitHub Actions: build wheels with compiled Cython
2name: Build
3on: [push]
4jobs:
5  build:
6    runs-on: ${{ matrix.os }}
7    strategy:
8      matrix:
9        os: [ubuntu-latest, macos-latest, windows-latest]
10        python: ['3.9', '3.10', '3.11', '3.12']
11    steps:
12      - uses: actions/checkout@v4
13      - uses: actions/setup-python@v5
14        with:
15          python-version: ${{ matrix.python }}
16      - run: pip install cython numpy
17      - run: pip install -e .
18      - run: pytest tests/

For distributing pre-compiled wheels across platforms, use cibuildwheel.

Common Pitfalls

Missing language_level=3: Without this directive, Cython defaults to Python 2 semantics in some cases (integer division, print as statement). Always set language_level='3' in compiler directives.
Not shipping .c files: If you only ship .pyx files, users must have Cython installed to build from source. Ship pre-generated .c files for maximum compatibility.
Import order in __init__.py: The try/except import pattern must import from the Cython module first, then fall back to pure Python. Getting this backward means you always use the slow path.
Forgetting build_ext --inplace during development: After changing .pyx files, you must recompile with python setup.py build_ext --inplace or pip install -e .. Stale .so files cause confusing behavior.
Type mismatch between Cython and Python fallback: Both implementations must accept the same arguments and return the same types. Write tests that run against both to catch discrepancies.

Summary

Place .pyx files in the package directory alongside .py files
Use setup.py with cythonize() to compile extensions during installation
Provide pure-Python fallbacks with try/except imports in __init__.py
Ship pre-generated .c files so users without Cython can still compile from source
Set language_level='3' in all Cython compiler directives
Test both the Cython and fallback implementations to ensure identical behavior

Tackling System Design Interview Problems

A short course that equips you with the skills to approach system design interviews methodically.

Start the free course

Basis Lab

No course covers this one yet

Basis Lab writes one for you from a sentence about what you want to be able to do, then teaches it and checks you understood. Your first course is free.

Build my course

Data Structures & Algorithms practice on Codemia

Step through 300+ coding interview problems with animated visualisers that show the data structure changing as the code runs. The Blind 75 are free.

Practice algorithms

How should I structure a Python package that contains Cython code

Data Structures & Algorithms practice on Codemia

Introduction

Recommended Directory Structure

pyproject.toml (Modern Build)

setup.py (Compilation Configuration)

The Cython Source (.pyx)

Pure-Python Fallback

init.py with Fallback Import

Building and Installing

Including Pre-Generated C Files

Cython Declaration Files (.pxd)

Testing Both Implementations

CI/CD Configuration

Common Pitfalls

Summary

Related reading

Tackling System Design Interview Problems

No course covers this one yet

Data Structures & Algorithms practice on Codemia