A3 Error Metrics for Predictive Models

Description

A package for the generation of accurate, accessible, and adaptable error metrics for developing high quality predictions and inferences. The name A3 (pronounced "A-Cubed") comes from the combination of the first letters of these three primary adjectives.

Details

The overarching purpose of the outputs and tools in this package are to make the accurate assessment of model errors more accessible to a wider audience. Furthermore, a standardized set of reporting features are provided by this package which create consistent outputs for virtually any predictive model. This makes it straightforward to compare, for instance, a linear regression model to more exotic techniques such as Random forests or Support vector machines.

The standard outputs for each model fit provided by the A3 package include:

• Average Slope: Equivalent to a linear regression coefficient.

• Cross Validated R^2: Robust calculation of R^2 (percent of squared error explained by the model compared to the null model) values adjusting for over-fitting.

• p Values: Robust calculation of p-values requiring no parametric assumptions other than independence between observations (which may be violated if compensated for).

The primary functions that will be used are a3 for arbitrary modeling functions and a3.lm for linear models. This package also includes print.A3 and plot.A3 for outputting the A3 results.

Author(s)

Scott Fortmann-Roe scottfr@berkeley.edu http://Scott.Fortmann-Roe.com

Results