In this article, we discuss how to calculate the Root Relative Squared Error in R?
The Root Relative Squared Error (RRSE) is a performance metric for predictive models, such as regression. It is a basic metric that gives a first indication of how well your model performance. Besides, it is an extension of the Relative Squared Error (RSE).
But, how do you calculate the RRSE?
The easiest way to calculate the Root Relative Squared Error (RRSE) in R is by using the RRSE() function from the Metrics packages. The RRSE() function requires two inputs, namely the realized and the predicted values, and, as a result, returns the Root Relative Squared Error.
Besides the RRSE() function, we discuss also 2 other methods to calculate the Root Relative Squared Error. One method requires only basic R code, whereas the other is slightly more complex but robust to missing values (NAs).
We support all explanations with examples and R code that you can use directly in your project. But, before we start, we first take a look at the definition of the Root Relative Squared Error.
The Root Relative Squared Error (RRSE)
Definition & Formula
The Root Relative Squared Error (RRSE) is defined as the square root of the sum of squared errors of a predictive model normalized by the sum of squared errors of a simple model. In other words, the square root of the Relative Squared Error (RSE).
, where:
- n: represents the number of observations
- yi: represents the realized value
- ŷi: represents the predicted value
- ȳ: represents the average of the realized values
Since the simple model is just the average of the realized values, the interpretation of the RRSE is simple. The Root Relative Squared Error indicates how well a model performs relative to the average of the true values. Therefore, when the RRSE is lower than one, the model performs better than the simple model. Hence, the lower the RRSE, the better the model.
Comparison RRSE, RAE, and RSE
Besides the RRSE, there also exist the Relative Absolute Error (RAE) and the Relative Squared Error (RSE). All three metrics measure how well a model performs relative to a simple model. However, there a minor differences.
The first difference is that the RAE assumes that the severity of errors increases linearly. For example, an error of 2 is twice as bad as an error of 1. However, the RSE and the RRSE put more emphasis on bigger errors.
The second difference is that the RAE and the RRSE measure the error in the same dimension as the amount predicted. Whereas, the RSE doesn’t use the same scale.
The table below summarizes the differences between the RAE, RSE, and RRSE
| RAE | RSE | RRSE | |
|---|---|---|---|
| Uses the same dimension as the amount predicted | X | X | |
| Puts more emphasis on bigger errors | X | X |
Numeric examples of the RAE, RSE, and RRSE
3 Ways to Calculate the Root Relative Squared Error in R
Before we show 3 methods to calculate the Root Relative Squared Error in R, we first create two vectors. These vectors represent the realized values (y) and predicted values (y_hat).
We use the SAMPLE.INT() and RNORM() functions to create the vectors.
set.seed(123)
y <- sample.int(100, 100, replace = TRUE)
y
set.seed(123)
y_hat <- y + rnorm(100, 0, 1)
y_hat1. Calculate the RRSE with Basic R Code
The first way to calculate the RRSE in R is by writing your own code. Since the definition of the RRSE is straightforward, you only need 3 functions to carry out the calculation, namely SQRT(), SUM(), and MEAN().
Although this method requires more code than the second method, this is our preferred method. Especially because it shows directly the definition of the RRSE which is useful for those not familiar with this metric.
Syntax
sqrt(sum((realized - predicted)**2) / sum((realized - mean(realized))**2))
Part of the Root Relative Squared Error is to square the errors, you can do this in R with the double-asterisk (**) or the caret symbol (^).
Example
sqrt(sum((y - y_hat)**2) / sum((y - mean(y))**2))
2. Calculate the RRSE with a Function from a Package
The easiest way to calculate the Root Relative Squared Error in R is by using the RRSE() function from the Metrics package. This function only requires the realized and the predicted values to calculate the Root Relative Squared Error.
Besides the RRSE() function, the Metrics package has many other functions to assess the performance of predictive models.
Syntax
rrse(actual, predicted)
Example
library(Metrics)
rrse(actual = y, predicted = y_hat)
3. Calculate the RRSE with a User-Defined Function
A disadvantage of the previous methods is the fact they are not robust to missing values. In other words, if the realized and/or the predicted values contains missing values, then both methods return a NA. See the example below.
y1 <- y
y1[1] <- NA
y_hat1 <- y_hat
y_hat1[1] <- NA
sqrt(sum((y1 - y_hat1)**2) / sum((y1 - mean(y1))**2))
rrse(actual = y1, predicted = y_hat1)
Therefore, a third method to calculate the Root Relative Squared Error is to create a user-defined function. Such as function requires more R code, but is robust to missing values. By adding the na.rm option, the SUM() and the MEAN() functions ignore missing values and prevent further problems.
Example
RootRelativeSquaredError <- function(realized, predicted, na.rm = TRUE){
error <- realized - predicted
mean_realized <- mean(realized, na.rm = na.rm)
sqrt(sum(error**2, na.rm = na.rm) / sum((realized - mean_realized)**2, na.rm = na.rm))
}
RootRelativeSquaredError(realized = y1, predicted = y_hat1)
