Introduction

MulticlassClassificationEvaluator in Spark ML gives aggregate metrics such as weighted precision and weighted recall, but it does not directly expose per-class precision and recall values. To get class-level metrics, use MulticlassMetrics from Spark MLlib on prediction-label pairs. This combination provides both summary and per-label diagnostics.

Core Sections

What MulticlassClassificationEvaluator Provides

Evaluator is great for model selection loops because it integrates with pipelines and parameter tuning.

1from pyspark.ml.evaluation import MulticlassClassificationEvaluator
2
3evaluator = MulticlassClassificationEvaluator(
4    labelCol="label",
5    predictionCol="prediction",
6    metricName="f1"
7)
8
9f1 = evaluator.evaluate(predictions)
10print("weighted f1:", f1)

But metrics like per-label precision are not directly returned.

Use MulticlassMetrics for Per-class Values

Convert prediction DataFrame to RDD of (prediction, label) pairs.

1from pyspark.mllib.evaluation import MulticlassMetrics
2
3prediction_and_labels = predictions.select("prediction", "label").rdd.map(
4    lambda row: (float(row.prediction), float(row.label))
5)
6
7metrics = MulticlassMetrics(prediction_and_labels)
8labels = metrics.labels
9
10for lbl in labels:
11    print(
12        "label", lbl,
13        "precision", metrics.precision(lbl),
14        "recall", metrics.recall(lbl),
15        "f1", metrics.fMeasure(lbl)
16    )

This yields true class-level diagnostics for imbalanced datasets.

Build a Report DataFrame

For easier downstream analysis, assemble per-class metrics into a Spark DataFrame.

1rows = [
2    (float(lbl), float(metrics.precision(lbl)), float(metrics.recall(lbl)), float(metrics.fMeasure(lbl)))
3    for lbl in labels
4]
5
6report_df = spark.createDataFrame(rows, ["label", "precision", "recall", "f1"])
7report_df.orderBy("label").show(truncate=False)

This report is useful in model monitoring and experiment tracking.

Compare Weighted and Per-class Metrics

Weighted scores can look strong even when minority classes perform poorly. Always inspect per-class metrics alongside weighted summaries.

For operational models, define class-specific thresholds and alerting based on business risk, not only global f1 values.

Integrate with Cross-validation Workflows

During hyperparameter tuning, keep weighted metric for selection speed, but run detailed per-class evaluation on best model candidate. This balances computational efficiency and decision quality.

Handle Label Indexing Carefully

If labels were transformed by indexers, map class indices back to original class names before reporting. Otherwise reports may be hard for stakeholders to interpret.

Confusion Matrix and Class Mapping

Per-class precision and recall become much more actionable when paired with confusion matrix analysis and readable class names.

cm = metrics.confusionMatrix().toArray()
print(cm)

If labels are numeric indices from StringIndexer, map them back:

label_names = indexer_model.labels
for i, name in enumerate(label_names):
    print(i, name)

This lets teams understand exactly which domain classes are being confused.

Reporting Workflow for Model Reviews

In model review reports, include weighted metrics, per-class table, confusion matrix, and class support counts together. A single metric is rarely enough for production decisions. For rare but high-risk classes, prioritize recall targets and monitor drift over time.

Automate this report generation in evaluation jobs so every model version is compared consistently. Standardized evaluation artifacts make approvals faster and more defensible.

Track per-class metrics over time in monitoring dashboards. Trend movement by label often reveals data drift earlier than aggregate scores.

Per-label thresholds and alerting should reflect business impact rather than relying on one global tolerance level.

Consistent evaluation templates improve comparability across experiments.

This approach supports stronger model-governance decisions in regulated environments.

Label-wise tracking also supports better retraining priorities.

Stable per-class reporting improves model accountability.

This consistency also improves cross-team communication.

Common Pitfalls

• Expecting evaluator metric names to include per-class precision directly.
• Forgetting to cast prediction and label values to float for MulticlassMetrics.
• Judging model quality only by weighted metrics on imbalanced datasets.
• Ignoring label-index mapping when presenting results to business teams.
• Running expensive per-class analysis in every tuning iteration unnecessarily.

Summary

• Spark evaluator provides aggregate multiclass metrics, not detailed per-label metrics.
• Use MulticlassMetrics for per-class precision, recall, and f1.
• Convert pair RDDs correctly and build structured reports.
• Compare weighted and class-level results for balanced model assessment.
• Map class indices back to domain labels for usable reporting.