Evaluation of multiple metrics and predictions

for a set of predictions from different models, evaluate multiple metrics and return the results in a tabular format that makes it easy to compare the predictions.

multieval(.dataset, .observed, .predictions, .metrics, value_table = FALSE)

Arguments

.dataset	data frame with the predictions, it must have at least the column with the observed data and at least one column that refers to the predictions of a model.
.observed	string with the name of the column that contains the observed data.
.predictions	string or vector of strings the columns where the predictions are stored.
.metrics	metric or set of metrics to be evaluated, the metrics refer to those allowed by the package 'yardstick' from 'tidymodels'.
value_table	TRUE to display disaggregated metrics.

Value

data frame with 4 columns: the evaluation metrics, the estimator used, the value of the metric and the name of the model.

Examples

set.seed(123)
library(yardstick) # métricas
#> For binary classification, the first factor level is assumed to be the event.
#> Use the argument `event_level = "second"` to alter this as needed.

predictions <-
  data.frame(truth = runif(100),
             predict_model_1 = rnorm(100, mean = 1,sd =2),
             predict_model_2 = rnorm(100, mean = 0,sd =2),
             predict_model_3 = rnorm(100, mean = 0,sd =3))

multieval(.dataset = predictions,
          .observed = "truth",
          .predictions = c("predict_model_1","predict_model_2","predict_model_3"),
          .metrics = list(rmse = rmse, rsq = rsq, mae = mae),
          value_table = TRUE)
#> $table_values
#> # A tibble: 9 x 4
#>   .metric .estimator .estimate modelo         
#>   <chr>   <chr>          <dbl> <chr>          
#> 1 rmse    standard   1.99      predict_model_1
#> 2 rmse    standard   1.95      predict_model_2
#> 3 rmse    standard   2.97      predict_model_3
#> 4 rsq     standard   0.000704  predict_model_1
#> 5 rsq     standard   0.00115   predict_model_2
#> 6 rsq     standard   0.0000892 predict_model_3
#> 7 mae     standard   1.59      predict_model_1
#> 8 mae     standard   1.61      predict_model_2
#> 9 mae     standard   2.37      predict_model_3
#> 
#> $summary_table
#> # A tibble: 3 x 4
#>   modelo           rmse       rsq   mae
#>   <chr>           <dbl>     <dbl> <dbl>
#> 1 predict_model_1  1.99 0.000704   1.59
#> 2 predict_model_2  1.95 0.00115    1.61
#> 3 predict_model_3  2.97 0.0000892  2.37
#> 

# Output ----------------------
# A tibble: 9 x 4
# .metric .estimator .estimate model
# <chr>   <chr>          <dbl> <chr>
#   1 mae     standard     1.45    predict_model_1
# 2 mae     standard     1.67    predict_model_2
# 3 mae     standard     2.43    predict_model_3
# 4 rmse    standard     1.78    predict_model_1
# 5 rmse    standard     2.11    predict_model_2
# 6 rmse    standard     3.01    predict_model_3
# 7 rsq     standard     0.00203 predict_model_1
# 8 rsq     standard     0.0158  predict_model_2
# 9 rsq     standard     0.00254 predict_model_3

#$summary_table
# A tibble: 3 x 4
#  model             mae  rmse     rsq
#  <chr>           <dbl> <dbl>   <dbl>
#  1 predict_model_1  1.45  1.78 0.00203
#  2 predict_model_2  1.67  2.11 0.0158
#  3 predict_model_3  2.43  3.01 0.00254

Evaluation of multiple metrics and predictions

Arguments

Value

See also

Examples