MFCC feature descriptors for audio classification using librosa

MFCC

audio classification

feature descriptors

librosa

machine learning

MFCC feature descriptors for audio classification using librosa

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Start Practicing Learn More

Introduction

Mel-Frequency Cepstral Coefficients (MFCCs) are the most widely used features for audio and speech classification tasks. They capture the timbral and spectral characteristics of audio in a compact representation that mimics how the human ear perceives sound. The Python library librosa provides efficient tools to extract MFCCs and related features from audio signals.

What Are MFCCs?

MFCCs are derived through a pipeline that transforms raw audio into a perceptually meaningful representation:

Frame the signal: Split audio into short overlapping windows (typically 20-40ms)
Compute the power spectrum: Apply FFT to each frame
Apply Mel filterbank: Map frequencies to the Mel scale (logarithmic, matching human hearing)
Take the log: Compute log energies for each Mel band
Apply DCT: Discrete Cosine Transform decorrelates the features, producing MFCCs

Typically 13 coefficients are used per frame, though 20-40 are sometimes extracted.

Extracting MFCCs with Librosa

Basic Extraction

python

1import librosa
2import numpy as np
3
4# Load audio file
5y, sr = librosa.load('audio_sample.wav', sr=22050)
6
7# Extract 13 MFCCs
8mfccs = librosa.feature.mfcc(y=y, sr=sr, n_mfcc=13)
9
10print(f"MFCC shape: {mfccs.shape}")
11# Output: (13, num_frames) — 13 coefficients per time frame

Customizing Parameters

python

1mfccs = librosa.feature.mfcc(
2    y=y,
3    sr=sr,
4    n_mfcc=13,         # number of coefficients
5    n_fft=2048,        # FFT window size
6    hop_length=512,    # step between frames
7    n_mels=128,        # number of Mel bands
8    fmin=20,           # minimum frequency (Hz)
9    fmax=8000          # maximum frequency (Hz)
10)

Aggregating Across Time

For classification, you typically need a fixed-length feature vector per audio clip. Aggregate across time frames:

python

1# Mean and standard deviation across time
2mfcc_mean = np.mean(mfccs, axis=1)    # shape: (13,)
3mfcc_std = np.std(mfccs, axis=1)      # shape: (13,)
4features = np.concatenate([mfcc_mean, mfcc_std])  # shape: (26,)

Delta and Delta-Delta Coefficients

These describe the velocity and acceleration of the MFCCs, providing temporal information akin to derivatives in calculus:

python

1# First derivative (delta) — captures rate of change
2mfcc_delta = librosa.feature.delta(mfccs)
3
4# Second derivative (delta-delta) — captures acceleration
5mfcc_delta2 = librosa.feature.delta(mfccs, order=2)
6
7# Stack all three for a richer feature set
8combined = np.vstack([mfccs, mfcc_delta, mfcc_delta2])
9# Shape: (39, num_frames) — 13 * 3 = 39 features per frame

Building an Audio Classifier

Feature Extraction Pipeline

python

1import os
2import librosa
3import numpy as np
4from sklearn.model_selection import train_test_split
5from sklearn.svm import SVC
6from sklearn.preprocessing import StandardScaler
7from sklearn.metrics import classification_report
8
9def extract_features(file_path, n_mfcc=13):
10    """Extract MFCC features from an audio file."""
11    y, sr = librosa.load(file_path, sr=22050, duration=5)
12
13    mfccs = librosa.feature.mfcc(y=y, sr=sr, n_mfcc=n_mfcc)
14    delta = librosa.feature.delta(mfccs)
15    delta2 = librosa.feature.delta(mfccs, order=2)
16
17    # Aggregate statistics
18    features = np.concatenate([
19        np.mean(mfccs, axis=1), np.std(mfccs, axis=1),
20        np.mean(delta, axis=1), np.std(delta, axis=1),
21        np.mean(delta2, axis=1), np.std(delta2, axis=1),
22    ])
23    return features  # shape: (78,) — 13 * 6
24
25# Load dataset
26features, labels = [], []
27for genre in ['rock', 'jazz', 'classical']:
28    for file in os.listdir(f'audio/{genre}'):
29        feat = extract_features(f'audio/{genre}/{file}')
30        features.append(feat)
31        labels.append(genre)
32
33X = np.array(features)
34y = np.array(labels)

Training and Evaluation

python

1X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)
2
3scaler = StandardScaler()
4X_train = scaler.fit_transform(X_train)
5X_test = scaler.transform(X_test)
6
7clf = SVC(kernel='rbf', C=10)
8clf.fit(X_train, y_train)
9
10print(classification_report(y_test, clf.predict(X_test)))

Contrast with Other Features

Comparing MFCCs with features like Chroma or Spectral Contrast can be insightful, depending on the application:

Feature	Best For	Captures
MFCC	Speech, general audio	Timbral/spectral shape
Chroma	Music (pitch/harmony)	Pitch class distribution
Spectral Contrast	Music genre	Peak-valley differences in spectrum
Mel Spectrogram	Deep learning input	Full time-frequency representation
Zero Crossing Rate	Speech vs music	Signal noisiness

Visualizing MFCCs

python

1import librosa.display
2import matplotlib.pyplot as plt
3
4plt.figure(figsize=(10, 4))
5librosa.display.specshow(mfccs, x_axis='time', sr=sr, hop_length=512)
6plt.colorbar(format='%+2.0f dB')
7plt.title('MFCCs')
8plt.tight_layout()
9plt.savefig('mfcc_plot.png')

Common Pitfalls

Not normalizing features: MFCCs have different scales across coefficients. Always apply StandardScaler or z-score normalization before feeding into classifiers.
Ignoring deltas: Using only static MFCCs loses temporal dynamics. Adding delta and delta-delta features typically improves classification by 5-15%.
Fixed duration assumption: Audio clips of different lengths produce different numbers of frames. Either pad/truncate to a fixed duration or use aggregation (mean, std) to get fixed-length vectors.
Sample rate mismatch: Librosa defaults to resampling at 22050 Hz. If your model was trained on 16000 Hz audio, set sr=16000 in librosa.load().
First coefficient: MFCC coefficient 0 represents overall energy and can dominate other features. Some pipelines drop it or normalize it separately.

Summary

MFCCs capture perceptually relevant spectral features from audio signals
Use librosa.feature.mfcc() to extract coefficients, typically 13 per frame
Add delta and delta-delta features for temporal context (39 features per frame)
Aggregate with mean/std across time for fixed-length feature vectors
Normalize features before classification and consider combining with other audio descriptors