The Nearest Neighbor Hot Deck Algorithms

Description

A comprehensive function that performs nearest neighbor hot deck imputation. Aspects such as variable weighting, distance types, and donor limiting are implemented. New concepts such as the optimal distribution of donors are also available.

Usage

impute.NN_HD(DATA = NULL, distance = "man", weights = "range", attributes = "sim",
 comp = "rw_dist", donor_limit = Inf, optimal_donor = "no",
 list_donors_recipients = NULL, diagnose = NULL)

Arguments

Details

argument: distance can be defined as:

• numeric matrix, donors x recipients distance matrix

• numeric length = 1, Minkovski parameter

• string length = 1, distance metric to be used:

  • "man", Manhattan distance

  • "eukl", Euclidean distance

  • "tscheb", Chebyshev distance

  • "mahal", Mahalanobis distance (covariance matrix calculated after missing data compensation, incompatible with comp="rw_dist")

argument: weights can be defined as:

• string length = 1, reweighting type "range", "sd", "var", or "none"

• numeric length = 1, one numeric weight for all variables

• string vector, like option 1, only different type for each variable (not implemented)

• numeric vector, like option 2, only different numeric weight for each variable

• list, mixture of options 3 and 4 (not implemented)

argument: comp can be defined as:

• "rw_dist", total distance is reweighted by number of distances that may be computed

• "mean", mean imputation is performed before distance calculation

• "rseq", random hot deck imputation, each variable sequentially (uses impute.SEQ_HD)

• "rsim", random hot deck imputation, each variable simultaneously (not implemented)

argument: donor_limit is a single number interpreted by its range:

• (0,1), dynamic donor limit, i.e., .5 means any 1 donor may serve up to 50% of all recipients, rounded up if fractional

• [1,Inf), static donor limit, i.e., 2 means any 1 donor may serve up to 2 recipients, fractional parts discarded

• Inf, no donor limit

argument: optimal_donor is a single string interpreted by its value:

• "no", donor-recipient matching is performed in order by which the recipients appear in the data (fastest)

• "rand", donor-recipient matching is performed in a random recipient-order

• "mmin", donor-recipient matching is performed by the matrix minimum method (sequence independent)

• "modifvam", donor-recipient matching is performed by a modified (only columns considered) Vogel's approximation method (sequence independent)

• "vam", donor-recipient matching is performed by the Vogel's approximation method (sequence independent)

• "odd", donor-recipient matching is performed via the optimal donor distribution method (sequence independent, best results)

argument: diagnose should be:

• NULL, no diagnostics will be returned.

• character string, desired variable name under which the diagnostics will be saved to .GlobalEnv. The following character strings will however default to NULL with a warning: "if", "else", "repeat", "while", "function", "for", "in", "next", "break", "TRUE", "FALSE", "NULL", "Inf", "NaN", "NA", "NA_integer_", "NA_real_", "NA_complex_", "NA_character_", "c", "q", "s", "t", "C", "D", "F", "I", "T"

• anything else, defaults to NULL with a warning.

Should be a character string of the desired variable name which will be created in .GlobalEnv

Value

An imputed data matrix the same size as the input DATA. If the diagnose option is used correctly, a list containing the following components will be created in the workspace:

Author(s)

Dieter William Joenssen Dieter.Joenssen@googlemail.com

References

Andridge, R.R. and Little, R.J.A. (2010) A Review of Hot Deck Imputation for Survey Non-response. International Statistical Review. 78, 40–64.

Bankhofer, U. and Joenssen, D.W. (2014) On Limiting Donor Usage for Imputation of Missing Data via Hot Deck Methods. In: M. Spiliopoulou, L. Schmidt-Thieme, and R. Jannings (Eds.): Data Analysis, Machine Learning and Knowledge Discovery. Studies in Classification, Data Analysis and Knowledge Organization, 3–11. Berlin/Heidelberg: Springer.

Domschke, W. (1995) Logistik: Transport. Munich: Oldenbourg. [in German]

Ford, B. (1983) An Overview of Hot Deck Procedures. In: W. Madow, H. Nisselson and I. Olkin (Eds.): Incomplete Data in Sample Surveys. New York: Academic Press, 185–207.

Joenssen, D.W. (2015) Donor Limited Hot Deck Imputation: A Constrained Optimization Problem. In: B. Lausen, S. Krolak-Schwerdt, and M. B"ohmer (Eds.): Data Science, Learning by Latent Structures, and Knowledge Discovery. Studies in Classification, Data Analysis and Knowledge Organization, pages 319–328. Berlin/Heidelberg: Springer.

Joenssen, D.W. (2015) Hot-Deck-Verfahren zur Imputation fehlender Daten – Auswirkungen des Donor-Limits. Ilmenau: Ilmedia. [in German, Dissertation]

Joenssen, D.W. and Bankhofer, U. (2012) Donor Limited Hot Deck Imputation: Effects on Parameter Estimation. Journal of Theoretical and Applied Computer Science. 6, 58–70.

Kalton, G. and Kasprzyk, D. (1986) The Treatment of Missing Survey Data. Survey Methodology. 12, 1–16.

Sande, I. (1983) Hot-Deck Imputation Procedures. In: W. Madow, H. Nisselson and I. Olkin (Eds.): Incomplete Data in Sample Surveys. New York: Academic Press, 339–349.

See Also

impute.mean, match.d_r_vam, reweight.data

Examples

#Set the random seed to an arbitrary number
set.seed(421)

#Generate random integer matrix size 10x4
Y<-matrix(sample(x=1:100,size=10*4),nrow=10)

#remove 5 values, ensuring one complete covariate and 5 donors
Y[-c(1:5),-1][sample(1:15,size=5)]<-NA

#Impute using various different (arbitrarily chosen) settings
impute.NN_HD(DATA=Y,distance="man",weights="var")

impute.NN_HD(DATA=Y,distance=2,weights=rep(.5,4),donor_limit=2,optimal_donor="mmin")

impute.NN_HD(DATA=Y,distance="eukl",weights=.25,comp="mean",donor_limit=1,
 optimal_donor="odd")
 
#Recover some diagnostics
impute.NN_HD(DATA=Y,distance="eukl",weights=.25,comp="mean",donor_limit=1,
 optimal_donor="odd",diagnose = "diagnostics")
# look at the diagnostics
 diagnostics

Results

R version 3.3.1 (2016-06-21) -- "Bug in Your Hair"
Copyright (C) 2016 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> library(HotDeckImputation)
Error in library(HotDeckImputation) : 
  there is no package called 'HotDeckImputation'
Execution halted