R Graphical Manual

Browse All

Last data update: 2014.03.03

R: Measures of Model Accuracy

accuracy

R Documentation

Measures of Model Accuracy

Description

accuracy estimates six measures of accuracy for presence-absence or presence-psuedoabsence data. These include AUC, ommission rates, sensitivity, specificity, proportion correctly identified and Kappa.

Note: this method will exclude any missing data.

Usage

accuracy(obs, pred, threshold = 0.5)

Arguments

`obs`	a vector of observed values which must be 0 for absences and 1 for occurrences
`pred`	a vector of the same length as `obs` representing the predicted values. Values must be between 0 & 1 prepresenting a likelihood.
`threshold`	this can be: a) a single value representing a single threshold between 0 & 1; b) a vector of threshold values between 0 & 1; OR c) an integer value representing the number of equal interval threshold values between 0 & 1

Value

a data.frame with seven columns:

`threshold`	the threshold values representing each row of data
`AUC`	the AUC given the defined threshold value
`ommission.rate`	the ommission rate as a proportion of true occurrences misidentified given the defined threshold value
`sensitivity`	the sensitivity given the defined threshold value
`specificity`	the specificity given the defined threshold value
`prop.correct`	the proportion of the presence and absence records correctly identified given the defined threshold value
`Kappa`	the Kappa statistic of the model given the defined threshold value

Author(s)

Jeremy VanDerWal jjvanderwal@gmail.com

Examples

#create some data
obs = c(sample(c(0,1),20,replace=TRUE),NA); obs = obs[order(obs)]
pred = runif(length(obs),0,1); pred = pred[order(pred)]

#calculate accuracy of the model with a single threshold value
accuracy(obs,pred,threshold=0.5)

#calculate accuracy given several defined thresholds
accuracy(obs,pred,threshold=c(0.33,0.5,0.66))

#calculate accuracy given a number of equal interval thresholds
accuracy(obs,pred,threshold=20)

Results


R version 3.3.1 (2016-06-21) -- "Bug in Your Hair"
Copyright (C) 2016 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> library(SDMTools)
> png(filename="/home/ddbj/snapshot/RGM3/R_CC/result/SDMTools/accuracy.Rd_%03d_medium.png", width=480, height=480)
> ### Name: accuracy
> ### Title: Measures of Model Accuracy
> ### Aliases: accuracy
> 
> ### ** Examples
> 
> #create some data
> obs = c(sample(c(0,1),20,replace=TRUE),NA); obs = obs[order(obs)]
> pred = runif(length(obs),0,1); pred = pred[order(pred)]
> 
> #calculate accuracy of the model with a single threshold value
> accuracy(obs,pred,threshold=0.5)
  threshold       AUC omission.rate sensitivity specificity prop.correct
1       0.5 0.7916667             0           1   0.5833333         0.75
      Kappa
1 0.5283019
Warning message:
In accuracy(obs, pred, threshold = 0.5) :
  1 data points removed due to missing data
> 
> #calculate accuracy given several defined thresholds
> accuracy(obs,pred,threshold=c(0.33,0.5,0.66))
  threshold       AUC omission.rate sensitivity specificity prop.correct
1      0.33 0.7083333             0           1   0.4166667         0.65
2      0.50 0.7916667             0           1   0.5833333         0.75
3      0.66 0.9166667             0           1   0.8333333         0.90
      Kappa
1 0.3636364
2 0.5283019
3 0.8000000
Warning message:
In accuracy(obs, pred, threshold = c(0.33, 0.5, 0.66)) :
  1 data points removed due to missing data
> 
> #calculate accuracy given a number of equal interval thresholds
> accuracy(obs,pred,threshold=20)
    threshold       AUC omission.rate sensitivity specificity prop.correct
1  0.00000000 0.5000000         0.000       1.000   0.0000000         0.40
2  0.05263158 0.5000000         0.000       1.000   0.0000000         0.40
3  0.10526316 0.5833333         0.000       1.000   0.1666667         0.50
4  0.15789474 0.6666667         0.000       1.000   0.3333333         0.60
5  0.21052632 0.7083333         0.000       1.000   0.4166667         0.65
6  0.26315789 0.7083333         0.000       1.000   0.4166667         0.65
7  0.31578947 0.7083333         0.000       1.000   0.4166667         0.65
8  0.36842105 0.7083333         0.000       1.000   0.4166667         0.65
9  0.42105263 0.7500000         0.000       1.000   0.5000000         0.70
10 0.47368421 0.7500000         0.000       1.000   0.5000000         0.70
11 0.52631579 0.7916667         0.000       1.000   0.5833333         0.75
12 0.57894737 0.8750000         0.000       1.000   0.7500000         0.85
13 0.63157895 0.9166667         0.000       1.000   0.8333333         0.90
14 0.68421053 1.0000000         0.000       1.000   1.0000000         1.00
15 0.73684211 0.9375000         0.125       0.875   1.0000000         0.95
16 0.78947368 0.8750000         0.250       0.750   1.0000000         0.90
17 0.84210526 0.8125000         0.375       0.625   1.0000000         0.85
18 0.89473684 0.7500000         0.500       0.500   1.0000000         0.80
19 0.94736842 0.5625000         0.875       0.125   1.0000000         0.65
20 1.00000000 0.5000000         1.000       0.000   1.0000000         0.60
       Kappa
1  0.0000000
2  0.0000000
3  0.1379310
4  0.2857143
5  0.3636364
6  0.3636364
7  0.3636364
8  0.3636364
9  0.4444444
10 0.4444444
11 0.5283019
12 0.7058824
13 0.8000000
14 1.0000000
15 0.8936170
16 0.7826087
17 0.6666667
18 0.5454545
19 0.1463415
20 0.0000000
Warning message:
In accuracy(obs, pred, threshold = 20) :
  1 data points removed due to missing data
> 
> 
> 
> 
> 
> dev.off()
null device 
          1 
>