Introduction

The neuralnet package does not ship with a built-in "relu" option the way it does for "logistic" and "tanh". It can accept a custom activation function through act.fct, but the package expects that function to be differentiable, which makes raw ReLU a slightly awkward fit compared with modern deep-learning frameworks.

What neuralnet Actually Supports

In neuralnet, the act.fct argument accepts:

• the string "logistic"
• the string "tanh"
• a custom differentiable R function

That means there is no native "relu" keyword. If you want ReLU-like behavior, you either provide a custom function or use a different library that treats ReLU as a first-class choice.

This matters because neuralnet is an older package aimed at relatively small feed-forward networks, not modern deep-learning workloads where ReLU is the default hidden-layer activation.

Why Raw ReLU Is Not Ideal Here

ReLU is defined as max(0, x). In practical machine learning, that is fine because frameworks define a usable derivative convention at zero. In neuralnet, the story is less smooth because the package expects a differentiable activation function and works best with functions it can handle cleanly during training.

So the answer is:

• 'neuralnet does not provide built-in ReLU'
• a custom function may be possible
• a smooth approximation is often safer in this package

If your goal is specifically “something ReLU-like,” softplus is often the better choice in neuralnet.

A Practical Alternative: Softplus

Softplus behaves like a smoothed ReLU:

1library(neuralnet)
2
3set.seed(42)
4
5softplus <- function(x) log(1 + exp(x))
6
7df <- data.frame(
8  x1 = runif(200, -1, 1),
9  x2 = runif(200, -1, 1)
10)
11
12df$y <- ifelse(df$x1 + df$x2 > 0, 1, 0)
13
14nn <- neuralnet(
15  y ~ x1 + x2,
16  data = df,
17  hidden = c(4, 2),
18  act.fct = softplus,
19  linear.output = FALSE
20)
21
22pred <- compute(nn, df[, c("x1", "x2")])
23head(pred$net.result)

This example is runnable, keeps the custom activation path, and stays closer to what neuralnet expects from an activation function.

If You Still Want to Try ReLU

You can experiment with a custom ReLU-style function, but the risk is training instability or derivative-handling issues depending on how the function is defined and how the package processes it.

For small educational experiments, the package can still be useful. For production work or real experimentation with ReLU-heavy networks, keras in R is a better tool.

Here is the equivalent idea with a framework that treats ReLU normally:

1library(keras)
2
3model <- keras_model_sequential() |>
4  layer_dense(units = 8, activation = "relu", input_shape = 2) |>
5  layer_dense(units = 1, activation = "sigmoid")
6
7model |>
8  compile(
9    optimizer = "adam",
10    loss = "binary_crossentropy",
11    metrics = "accuracy"
12  )

That is the cleaner route if the question is really about using ReLU as a standard network component rather than forcing it into an older package.

When neuralnet Is Still Fine

neuralnet is still useful for:

• small tabular experiments
• teaching feed-forward network concepts
• quick prototypes with logistic or tanh activations

It is less attractive when you need:

• ReLU and its common variants
• large datasets
• convolutional or recurrent layers
• modern optimizer and callback support

That boundary is worth stating explicitly so the tool choice matches the problem.

Common Pitfalls

The biggest mistake is assuming act.fct = "relu" is a built-in option. It is not.

Another issue is treating a raw piecewise ReLU definition as if every R neural-network package will handle it like TensorFlow or PyTorch. neuralnet is older and expects custom activation functions to fit its differentiation model.

Developers also sometimes set linear.output incorrectly. If you want the activation applied at the output layer, it must be FALSE; if you want a linear output layer, keep it TRUE.

Finally, do not force an outdated package into a job it was not designed for. If ReLU is central to the model design, switching libraries is often the more honest engineering decision.

Summary

• 'neuralnet has built-in "logistic" and "tanh", not built-in ReLU.'
• It can accept a custom activation function through act.fct.
• Because the package expects differentiable custom functions, softplus is often safer than raw ReLU.
• For modern ReLU-based neural networks, keras in R is usually the better choice.
• Choose the library based on the model you actually want to train.