Introduction

The ROCR package in R makes it easy to compute ROC curves and the area under the curve, usually called AUC, for binary classifiers. The key requirement is to pass score-like predictions, such as probabilities or decision values, rather than hard class labels.

Build a prediction Object First

ROCR starts from a prediction object created from model scores and true labels. The two vectors must have the same length.

1library(ROCR)
2
3scores <- c(0.91, 0.84, 0.72, 0.61, 0.55, 0.42, 0.31, 0.12)
4labels <- c(1, 1, 1, 0, 1, 0, 0, 0)
5
6pred <- prediction(predictions = scores, labels = labels)

Use probabilities, margins, or confidence scores here. If you pass only 0 and 1 class predictions, the ROC curve becomes much less informative because there are almost no thresholds to sweep over.

Compute ROC and AUC

Once you have the prediction object, use performance() to ask for different metrics. For AUC:

1auc_perf <- performance(pred, measure = "auc")
2auc_value <- as.numeric(auc_perf@y.values[[1]])
3
4print(auc_value)

To build the ROC curve itself:

1roc_perf <- performance(pred, measure = "tpr", x.measure = "fpr")
2
3plot(
4  roc_perf,
5  col = "blue",
6  lwd = 2,
7  main = paste("ROC Curve, AUC =", round(auc_value, 3))
8)
9
10abline(a = 0, b = 1, col = "gray", lty = 2)

That diagonal line represents random guessing, so a model with real signal should curve above it.

Wrap the Pattern in a Helper

If you compare several models, a helper function keeps the workflow consistent.

1library(ROCR)
2
3compute_auc <- function(scores, labels) {
4  pred <- prediction(scores, labels)
5  perf <- performance(pred, "auc")
6  as.numeric(perf@y.values[[1]])
7}
8
9model_a <- c(0.90, 0.80, 0.70, 0.40, 0.30, 0.20)
10model_b <- c(0.85, 0.82, 0.65, 0.50, 0.28, 0.18)
11truth <- c(1, 1, 1, 0, 0, 0)
12
13print(compute_auc(model_a, truth))
14print(compute_auc(model_b, truth))

That is especially useful in cross-validation workflows where you want one AUC value per fold.

Interpret AUC Carefully

AUC is threshold-independent, which makes it attractive for comparing ranking quality. But it does not tell you everything about model usefulness. In imbalanced problems, a model can have a decent AUC and still perform poorly at the decision threshold that matters in practice.

For that reason, it is common to pair AUC with:

• precision-recall analysis
• confusion matrices at specific thresholds
• domain-specific cost metrics

If your business cost is dominated by false positives or false negatives, AUC alone is not enough.

Common Pitfalls

The biggest mistake is passing hard class predictions instead of continuous scores. AUC is about ranking and threshold movement, so it needs more than a final yes-or-no output.

Another common issue is accidentally reversing the positive and negative classes. If label encoding changes across experiments, the AUC comparison becomes meaningless.

People also compare AUC values from different datasets or different cross-validation splits as if they were directly equivalent. Consistent evaluation data matters more than tiny differences in code.

Finally, remember that ROCR is for binary classification in this workflow. Multiclass problems need a one-vs-rest or similar evaluation strategy before you interpret AUC sensibly.

If you report AUC to other teams, include the evaluation dataset and positive-class definition alongside the number. Without that context, the metric is easy to misread.

Summary

• Create a prediction object from score-like predictions and true binary labels.
• Compute AUC with performance(pred, "auc").
• Plot ROC with performance(pred, "tpr", x.measure = "fpr").
• Use probability-like scores, not hard class labels.
• Pair AUC with other metrics when threshold behavior or class imbalance matters.