Precision and Recall are fundamental metrics used to evaluate the performance of models, especially in information retrieval and binary classification contexts. To strike a balance between these two metrics, we use the F-Measure (also known as the F-Score or F1-Score). This article delves into why the F-Measure is defined as the harmonic mean and not the arithmetic mean of Precision and Recall.

Definitions

Before we dive into the technical details, let's briefly define Precision and Recall:

• Precision: The ratio of correctly predicted positive observations to the total predicted positives. It is formulated as:
  $$\text{Precision} = \frac{\text{True Positives (TP)}}{\text{True Positives (TP)} + \text{False Positives (FP)}}$$- Recall: The ratio of correctly predicted positive observations to all the actual positives. It is formulated as:
  $$\text{Recall} = \frac{\text{True Positives (TP)}}{\text{True Positives (TP)} + \text{False Negatives (FN)}}$$## Understanding Arithmetical and Harmonic Means

Arithmetic Mean

The arithmetic mean is the sum of values divided by the count of values. While it is a straightforward way of averaging, it does not handle ratios or rates well. An arithmetic average assumes that each component holds equal weight, which may not hold true in the context of precision and recall.

Harmonic Mean

The harmonic mean is defined as the reciprocal of the arithmetic mean of the reciprocals. In the context of two values $a$ and $b$, the harmonic mean is:

$$HM = \frac{2ab}{a + b}$$The harmonic mean is more appropriate for rates (like precision and recall) as it tends to favor balanced rates, providing a score that favors neither over-prediction nor under-prediction excessively.

Why Use the Harmonic Mean for F-Measure?

The F-Measure seeks a balance between precision and recall. Here's why the harmonic mean is advantageous:

1. Sensitivity to Imbalances: The harmonic mean is sensitive to the differences in magnitude between precision and recall. For example, if either precision or recall is extremely low, the harmonic mean penalizes the F-score significantly, reflecting the inefficiency effectively. This property is crucial as both precision and recall should generally be high for a good model.
2. Handling Proportions: Precision and recall are rates (proportions) akin to speed (e.g., miles per hour). When aggregating rates, the harmonic mean is more appropriate because it naturally accounts for the ratios involved.
3. Discounting Extreme Values: The arithmetic mean would overly favor higher rates, while the harmonic mean provides a more conservative result.

Technical Example

Consider a binary classification scenario:

• System A: Precision = 0.9, Recall = 0.1
• System B: Precision = 0.5, Recall = 0.5
• Arithmetic Mean of A: $\frac{0.9 + 0.1}{2} = 0.5$
• Harmonic Mean (F1-Score) of A: $\frac{2 \times 0.9 \times 0.1}{0.9 + 0.1} = 0.18$
• Arithmetic Mean of B: $\frac{0.5 + 0.5}{2} = 0.5$
• Harmonic Mean (F1-Score) of B: $\frac{2 \times 0.5 \times 0.5}{0.5 + 0.5} = 0.5$

The arithmetic mean poorly distinguishes between a balanced system and one with an extreme imbalance. The harmonic mean, however, appropriately reflects the low performance of System A due to its poor recall.

Summary Table

Conclusion

Human decision-making in model evaluation relies on a comprehensive measure that balances precision and recall effectively. The harmonic mean, used in the F-Measure, prioritizes balance and provides a realistic assessment when dealing with rates like precision and recall. While the arithmetic mean might seem intuitive, the harmonic mean’s ability to appropriately weigh imbalances makes it ideal for summarizing the performance of classification models.