hierarchical clustering with gene expression matrix in python
ML System Design practice on Codemia
Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.
Hierarchical clustering is a valuable technique in bioinformatics, often applied to analyze gene expression data. It organizes data into a dendrogram, offering insights into gene or sample similarities. This article delves into hierarchical clustering using a gene expression matrix in Python, covering technical aspects, practical examples, implementation, and interpretation of results.
Introduction to Hierarchical Clustering
Hierarchical clustering groups similar data points into clusters, forming a hierarchical tree-like structure termed a dendrogram. Unlike k-means clustering, which requires pre-specifying the number of clusters, hierarchical clustering unveils the data's natural clustering tendency without this prerequisite.
Types of Hierarchical Clustering
- Agglomerative Clustering (Bottom-Up Approach): Each data point starts as its own cluster, then iteratively merged based on similarity.
- Divisive Clustering (Top-Down Approach): It starts with a single cluster encompassing all data points, splitting it iteratively into smaller clusters.
This guide primarily focuses on agglomerative hierarchical clustering due to its prevalent use in gene expression analysis.
Gene Expression Matrix
A gene expression matrix consists of genes (rows) and samples (columns), with each cell representing the expression level of a gene in a sample. Analyzing such matrices unveils genes with similar expression patterns or identifies groups of samples with similar responses.
Example Data
Let's assume a gene expression matrix with 5 genes and 4 samples. This fictional dataset serves as the groundwork for our analysis.
| Gene | Sample1 | Sample2 | Sample3 | Sample4 |
| Gene1 | 6.5 | 2.3 | 8.5 | 5.4 |
| Gene2 | 7.1 | 3.7 | 7.9 | 4.8 |
| Gene3 | 5.9 | 3.1 | 8.6 | 5.6 |
| Gene4 | 6.8 | 2.6 | 7.5 | 5.3 |
| Gene5 | 7.4 | 3.0 | 8.1 | 5.1 |
Performing Hierarchical Clustering in Python
Hierarchical clustering can be conducted using libraries such as SciPy or scikit-learn.
Step-by-Step Guide
1. Import Libraries
- Leaves denote individual genes.
- Branches portray clusters, with shorter branches representing higher similarity.
- The height of the branches reflects the distance where clusters join.
Related reading
- High AUC but bad predictions with imbalanced data
- High bias convolutional neural network not improving with more layers/filters
- High volume SVM machine learning system
- Higher validation accuracy, than training accurracy using Tensorflow and Keras
- High performance fuzzy string comparison in Python, use Levenshtein or difflib
- Hopcroft–Karp algorithm in Python
- Higher validation accuracy, than training accurracy using Tensorflow and Keras
- HMM algorithm for gesture recognition
.png&w=3840&q=75)
Tackling System Design Interview Problems
A short course that equips you with the skills to approach system design interviews methodically.
Start the free courseTrack what you have practised
A free account saves your progress, solutions and study plan across every problem on Codemia.
ML System Design practice on Codemia
Design recommenders, ranking systems and training pipelines the way ML interviews actually ask for them, with worked solutions.