Introduction

Pruning a decision tree in R means trimming back branches that improve training accuracy but are unlikely to generalize well on new data. Most tree packages let you grow a tree first and then choose a simpler version using a complexity measure or validation result. In day-to-day R work, the standard pruning workflow uses rpart, its CP table, and a final call to prune().

Why Trees Need Pruning

Decision trees are easy to interpret, but they also overfit easily. If the tree keeps splitting until leaves are tiny, it often learns noise rather than stable structure.

Pruning helps by:

• reducing overfitting
• improving interpretability
• lowering model variance
• making the final tree easier to explain

There are two broad styles:

• pre-pruning, which stops growth early
• post-pruning, which grows a larger tree and then cuts it back

In R, post-pruning is often easier to inspect because you can compare candidate trees before choosing one.

Build an Initial Tree With rpart

Start with a fitted model.

1library(rpart)
2
3fit <- rpart(
4  Species ~ Sepal.Length + Sepal.Width + Petal.Length + Petal.Width,
5  data = iris,
6  method = "class"
7)
8
9print(fit)
10plot(fit)
11text(fit, use.n = TRUE, cex = 0.8)

This gives you an initial tree. Depending on the dataset, it may already be larger than you want for production or explanation.

Inspect the Complexity Parameter Table

rpart stores pruning candidates in the complexity-parameter table.

printcp(fit)
plotcp(fit)

The table includes:

• 'CP, the pruning candidate'
• 'nsplit, the number of splits'
• 'xerror, the cross-validated relative error'
• 'xstd, the variability of that error estimate'

This table is the main tool for deciding how much of the tree to cut back.

Prune With the Best CP Value

One common strategy is to choose the row with the smallest cross-validated error and prune there.

1cp_table <- fit$cptable
2best_row <- which.min(cp_table[, "xerror"])
3best_cp <- cp_table[best_row, "CP"]
4
5pruned_fit <- prune(fit, cp = best_cp)
6
7printcp(pruned_fit)
8plot(pruned_fit)
9text(pruned_fit, use.n = TRUE, cex = 0.8)

This creates a smaller tree chosen from the path that rpart already computed.

The One-Standard-Error Rule

Choosing the absolute minimum xerror is not the only reasonable option. A common alternative is the one-standard-error rule: choose the simplest tree whose error is within one standard error of the minimum.

1cp_table <- fit$cptable
2min_row <- which.min(cp_table[, "xerror"])
3threshold <- cp_table[min_row, "xerror"] + cp_table[min_row, "xstd"]
4
5candidate_rows <- which(cp_table[, "xerror"] <= threshold)
6chosen_row <- min(candidate_rows)
7chosen_cp <- cp_table[chosen_row, "CP"]
8
9pruned_1se <- prune(fit, cp = chosen_cp)

This often gives you a slightly simpler tree with nearly the same predictive quality.

Evaluate Before and After Pruning

Do not assume the pruned tree is automatically better. Compare results on held-out data.

1set.seed(42)
2idx <- sample(seq_len(nrow(iris)), size = 0.7 * nrow(iris))
3train <- iris[idx, ]
4test <- iris[-idx, ]
5
6fit <- rpart(Species ~ ., data = train, method = "class")
7best_cp <- fit$cptable[which.min(fit$cptable[, "xerror"]), "CP"]
8pruned_fit <- prune(fit, cp = best_cp)
9
10pred_full <- predict(fit, test, type = "class")
11pred_pruned <- predict(pruned_fit, test, type = "class")
12
13mean(pred_full == test$Species)
14mean(pred_pruned == test$Species)

Sometimes pruning improves generalization, and sometimes it mainly improves simplicity with little accuracy change. Both outcomes can be useful.

Pre-Pruning Controls

You can also limit tree growth up front with rpart.control().

1fit_small <- rpart(
2  Species ~ .,
3  data = iris,
4  method = "class",
5  control = rpart.control(maxdepth = 3, minsplit = 10)
6)

Useful controls include:

• 'cp'
• 'maxdepth'
• 'minsplit'
• 'minbucket'

Pre-pruning can help, but it should not replace validation-based pruning if model selection matters.

Common Pitfalls

• Choosing the smallest CP value blindly. Fix: inspect cross-validated error and compare against simpler trees.
• Judging the tree only by training accuracy. Fix: validate on held-out data.
• Ignoring the CP table. Fix: use printcp() and plotcp() before deciding where to prune.
• Assuming a bigger tree is always more accurate. Fix: remember that extra splits often fit noise.
• Using pruning without considering interpretability goals. Fix: choose the final tree based on both performance and explainability needs.

Summary

• Pruning reduces tree complexity and often improves generalization.
• In R, rpart plus prune() is the usual workflow.
• Use the CP table to choose a pruning point instead of guessing.
• The one-standard-error rule is a good option when you prefer simpler trees.
• Always compare pruned and unpruned trees on held-out data.