Simple Python implementation of collaborative topic modeling?

Python

Collaborative Topic Modeling

Machine Learning

Topic Modeling

Python Programming

Simple Python implementation of collaborative topic modeling?

ML System Design practice on Codemia

Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.

Practice ML system design

Machine Learning Python

Introduction

Collaborative Topic Modeling (CTM) combines collaborative filtering (user-item interactions) with topic modeling (text content analysis) to make recommendations. It uses Latent Dirichlet Allocation (LDA) to extract topics from item descriptions, then integrates those topics with user preference vectors through matrix factorization. This hybrid approach handles the cold-start problem better than pure collaborative filtering.

How CTM Works

CTM has two components that work together:

Topic Model (LDA): Extracts topic distributions from item content (articles, product descriptions, papers)
Collaborative Filter (Matrix Factorization): Learns user preference vectors that align with topic distributions

The key insight: instead of learning arbitrary latent factors for items (as in standard matrix factorization), CTM constrains item factors to be close to their LDA topic distributions.

Step 1: Prepare the Data

python

1import numpy as np
2from sklearn.decomposition import LatentDirichletAllocation
3from sklearn.feature_extraction.text import CountVectorizer
4
5# User-item interaction matrix (ratings or binary)
6# Rows = users, Columns = items
7R = np.array([
8    [5, 3, 0, 1, 0],
9    [4, 0, 0, 1, 0],
10    [1, 1, 0, 5, 0],
11    [0, 0, 5, 4, 4],
12    [0, 1, 4, 0, 5],
13])
14
15# Item descriptions (one per item)
16item_texts = [
17    "machine learning neural networks deep learning AI",
18    "data science statistics regression analysis",
19    "cooking recipes Italian food pasta",
20    "artificial intelligence robotics automation",
21    "baking desserts cake chocolate"
22]

Step 2: Extract Topics with LDA

python

1# Convert text to document-term matrix
2vectorizer = CountVectorizer(stop_words='english')
3doc_term_matrix = vectorizer.fit_transform(item_texts)
4
5# Fit LDA to extract topic distributions
6n_topics = 3
7lda = LatentDirichletAllocation(
8    n_components=n_topics,
9    random_state=42,
10    max_iter=50
11)
12lda.fit(doc_term_matrix)
13
14# Topic distribution for each item (items x topics)
15theta = lda.transform(doc_term_matrix)
16print("Item-topic distributions:")
17print(theta.round(3))
18# Each row sums to 1 — probability of each topic per item

Step 3: Collaborative Topic Model

python

1class CollaborativeTopicModel:
2    def __init__(self, n_topics, n_users, n_items, lambda_u=0.1, lambda_v=10.0):
3        self.n_topics = n_topics
4        self.n_users = n_users
5        self.n_items = n_items
6        self.lambda_u = lambda_u   # Regularization for user factors
7        self.lambda_v = lambda_v   # Weight of topic prior for item factors
8
9    def fit(self, R, theta, n_iter=100, learning_rate=0.01):
10        """
11        R: user-item interaction matrix (n_users x n_items)
12        theta: LDA topic distributions (n_items x n_topics)
13        """
14        # Initialize user and item latent factors
15        self.U = np.random.normal(0, 0.1, (self.n_users, self.n_topics))
16        self.V = theta.copy()  # Initialize item factors from LDA topics
17
18        mask = R > 0  # Only train on observed interactions
19
20        for iteration in range(n_iter):
21            # Update user factors
22            for u in range(self.n_users):
23                observed = mask[u]
24                if not observed.any():
25                    continue
26                V_obs = self.V[observed]
27                R_obs = R[u, observed]
28
29                # Solve: (V^T V + lambda_u * I) * U_u = V^T * R_u
30                A = V_obs.T @ V_obs + self.lambda_u * np.eye(self.n_topics)
31                b = V_obs.T @ R_obs
32                self.U[u] = np.linalg.solve(A, b)
33
34            # Update item factors (pulled toward LDA topics)
35            for i in range(self.n_items):
36                observed = mask[:, i]
37                if not observed.any():
38                    # No interactions — use pure LDA topic distribution
39                    self.V[i] = theta[i]
40                    continue
41                U_obs = self.U[observed]
42                R_obs = R[observed, i]
43
44                # Solve: (U^T U + lambda_v * I) * V_i = U^T * R_i + lambda_v * theta_i
45                A = U_obs.T @ U_obs + self.lambda_v * np.eye(self.n_topics)
46                b = U_obs.T @ R_obs + self.lambda_v * theta[i]
47                self.V[i] = np.linalg.solve(A, b)
48
49            # Compute loss
50            if iteration % 20 == 0:
51                pred = self.U @ self.V.T
52                loss = np.sum(mask * (R - pred) ** 2)
53                reg = self.lambda_u * np.sum(self.U ** 2) + \
54                      self.lambda_v * np.sum((self.V - theta) ** 2)
55                print(f"Iter {iteration}: loss={loss:.4f}, reg={reg:.4f}")
56
57    def predict(self, user_idx, item_idx):
58        return self.U[user_idx] @ self.V[item_idx]
59
60    def recommend(self, user_idx, n=5):
61        scores = self.U[user_idx] @ self.V.T
62        top_items = np.argsort(scores)[::-1][:n]
63        return top_items, scores[top_items]

Step 4: Train and Predict

python

1# Train the model
2ctm = CollaborativeTopicModel(
3    n_topics=n_topics,
4    n_users=R.shape[0],
5    n_items=R.shape[1],
6    lambda_u=0.1,
7    lambda_v=10.0
8)
9ctm.fit(R, theta, n_iter=100)
10
11# Get recommendations for user 0
12recommended_items, scores = ctm.recommend(user_idx=0, n=3)
13print(f"Recommended items for user 0: {recommended_items}")
14print(f"Scores: {scores.round(3)}")
15
16# Predict a specific rating
17predicted = ctm.predict(user_idx=0, item_idx=2)
18print(f"Predicted rating for user 0, item 2: {predicted:.2f}")

Step 5: Evaluate

python

1from sklearn.metrics import mean_squared_error
2
3# Evaluate on observed entries
4mask = R > 0
5predictions = ctm.U @ ctm.V.T
6observed_actual = R[mask]
7observed_predicted = predictions[mask]
8
9rmse = np.sqrt(mean_squared_error(observed_actual, observed_predicted))
10print(f"RMSE on observed ratings: {rmse:.4f}")

CTM vs Standard Matrix Factorization

Feature	Standard MF	CTM
Cold start	Cannot recommend new items	Uses LDA topics for new items
Content awareness	No	Yes — uses item text
Interpretability	Latent factors are opaque	Factors correspond to topics
Data needed	Only user-item matrix	User-item matrix + item text

Common Pitfalls

Lambda balance: lambda_v controls how strongly item factors are pulled toward LDA topics. Too high and the model ignores user preferences. Too low and it becomes standard matrix factorization. Tune with cross-validation.
LDA quality: CTM is only as good as its topic model. If LDA produces poor topics (too few or too many), the recommendations suffer. Tune n_topics using coherence scores.
Sparse interactions: With very few ratings per user, the user factors are poorly estimated. Increase lambda_u regularization for sparse data.
Scaling: The alternating least squares update loop is O(n_users * n_items * n_topics). For large datasets, use stochastic gradient descent or batch updates instead of full ALS.
Missing vs zero: In the interaction matrix, 0 should mean "not observed," not "user dislikes item." Only train on observed entries (where mask = R > 0).

Summary

CTM combines LDA topic modeling with matrix factorization for content-aware recommendations
Item latent factors are regularized toward their LDA topic distributions
This handles the cold-start problem — new items with text but no ratings get topic-based factors
Use alternating least squares to update user and item factors iteratively
Tune lambda_v to balance between content-based and collaborative signals

Tackling System Design Interview Problems

A short course that equips you with the skills to approach system design interviews methodically.

Start the free course

Track what you have practised

A free account saves your progress, solutions and study plan across every problem on Codemia.

ML System Design practice on Codemia

Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.

Practice ML system design

Simple Python implementation of collaborative topic modeling?

ML System Design practice on Codemia

Introduction

How CTM Works

Step 1: Prepare the Data

Step 2: Extract Topics with LDA

Step 3: Collaborative Topic Model

Step 4: Train and Predict

Step 5: Evaluate

CTM vs Standard Matrix Factorization

Common Pitfalls

Summary

Related reading

Tackling System Design Interview Problems

Track what you have practised

ML System Design practice on Codemia