Model-Free Selection of Covariate Sets

Description

Dimension reduction of the covariate vector under unconfoundedness using model-free backward elimination algorithms, based on either marginal co-ordinate hypothesis testing, (MCH), (continuous covariates only) or kernel-based smoothing, (KS).

Usage

cov.sel(T, Y, X, type=c("dr", "np"), alg = 3,scope = NULL, alpha = 0.1,
		thru=0.5,thro=0.25,thrc=100,...)

Arguments

Details

Performs model-free selection of covariates for situations where the parameter of interest is an average causal effect. This function is based on the framework of sufficient dimension reduction, that under unconfoundedness, reduces dimension of the covariate vector. A two-step procedure searching for a sufficient subset of the covariate vector is implemented in the form of algorithms. This function uses MCH (if type="dr") or KS (if type="np") in the form of two backward elimination algorithms, Algorithm A and Algorithm B proposed by de Luna, Waernbaum and Richardson (2011).

Algorithm A (alg = 1): First the covariates conditionally independent of the treatment, T, given the rest of the variables (X.T) are removed. Then the covariates conditionally independent of the potential outcomes (in each of the treatment groups) given the rest of the covariates are removed. This yields two subsets of covariates; Q.1 and Q.0 for the treatment and control group respectively.

Algorithm B (alg = 2): First the covariates conditionally independent of the potential outcome (in each of the treatment groups), given the rest of the covariates (X.0 and X.1) are removed. Then the covariates conditionally independent of the treatment, T, given the rest of the covariates are removed. This yields two subsets of covariates; Z.1 and Z.0 for the treatment and control group respectively.

alg=3 runs both Algorithm A and B.

In KS the bandwidth range for unordered discrete covariates is [0, 1/#levels] while for ordered discrete covariates, no matter how many levels, the range is [0, 1]. For continuous covariates bandwidths ranges from 0 to infinity. Ordered discrete and continuous covariates are removed if their bandwidths exceed their respective thresholds. Unordered discrete covariates are removed if their bandwidths are larger than thru times the maximum bandwidth.

In case of MCH one can choose between sliced inverse regression, SIR, or sliced average variance estimation, SAVE. For KS the regression type can be set to local constant kernel or local linear and the bandwidth type can be set to fixed, generalized nearest neighbors or adaptive nearest neighbors. See dr and npregbw for details. Since type="np" results in a fully nonparametric covariate selection procedure this can be much slower than if type="dr".

Value

cov.sel returns a list with the following content:

If type="dr" the following type-specific content is returned:

If type="np" the following type-specific content is returned:

For marginal co-ordinate hypothesis test, type="dr", as a side effect a data frame of labels, tests, and p.values is printed.

Note

cov.sel calls the functions dr, dr.step and npregbw so the packages dr and np are required.

Author(s)

Emma Persson, <emma.persson@umu.se>, Jenny H<c3><83><c2><a4>ggstr<c3><83><c2><b6>m, <jenny.haggstrom@umu.se>

References

Cook, R. D. (2004). Testing Predictor contributions in Sufficient Dimension Reduction. The Annals of statistics 32. 1061-1092

de Luna, X., I. Waernbaum, and T. S. Richardson (2011). Covariate selection for the nonparametric estimation of an average treatment effect. Biometrika 98. 861-875

H<c3><83><c2><a4>ggstr<c3><83><c2><b6>m, J., E. Persson, I. Waernbaum and X. de Luna (2015). An R Package for Covariate Selection When Estimating Average Causal Effects. Journal of Statistical Software 68. 1-20

Hall, P., Q. Li and J.S. Racine (2007). Nonparametric estimation of regression functions in the presence of irrelevant regressors. The Review of Economics and Statistics, 89. 784-789

Li, L., R. D. Cook, and C. J. Nachtsheim (2005). Model-free Variable Selection. Journal of the Royal Statistical Society, Series B 67. 285-299

See Also

dr, np

Examples

## Marginal co-ordinate hypothesis test, continuous covariates only

data(datc)


##Algorithm A, keeping x6 and x7
  
ans <- cov.sel(T = datc$T, Y = datc$y, X = datc[,1:8], type="dr",
               alpha = 0.1, alg = 1, scope=c("x6","x7"))

summary(ans)

##Algorithm B, method "save"
  
ans <- cov.sel(T = datc$T, Y = datc$y, X = datc[,1:10], type="dr",
               alg = 2, method = "save", alpha = 0.3, na.action = "na.omit")

## Kernel-based smoothing, both categorical and continuous covariates

data(datfc)
##The example below with default setting takes about 9 minutes to run.
## ans <- cov.sel(T = datfc$T, Y = datfc$y, X = datfc[,1:8], type="np",
##               alpha = 0.1, alg = 3, scope=NULL, thru=0.5, thro=0.25, thrc=100)

## For illustration purposes we run Algorithm A using only the first 100 observations 
##and x1, x2, x3, x4 in datfc
ans <- cov.sel(T = datfc$T[1:100], Y = datfc$y[1:100], X = datfc[1:100,1:4], 
      type="np",alpha = 0.1, alg = 1, scope=NULL, thru=0.5, thro=0.25, thrc=100)

##The example below running Algorithm A, keeping x6 and x7 with regtype="ll"
##takes about 7 minutes to run.
##ans <- cov.sel(T = datfc$T, Y = datfc$y, X = datfc[,1:8], type="np",
##               alpha = 0.1, alg = 3, scope=c("x6","x7"), thru=0.5, thro=0.25, 
##               thrc=100, regtype="ll")

Results

R version 3.3.1 (2016-06-21) -- "Bug in Your Hair"
Copyright (C) 2016 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> library(CovSel)
Loading required package: dr
Loading required package: MASS
Loading required package: np
Nonparametric Kernel Methods for Mixed Datatypes (version 0.60-2)
[vignette("np_faq",package="np") provides answers to frequently asked questions]
> png(filename="/home/ddbj/snapshot/RGM3/R_CC/result/CovSel/cov.sel.Rd_%03d_medium.png", width=480, height=480)
> ### Name: cov.sel
> ### Title: Model-Free Selection of Covariate Sets
> ### Aliases: cov.sel
> 
> ### ** Examples
> 
> ## Marginal co-ordinate hypothesis test, continuous covariates only
> 
> data(datc)
> 
> 
> ##Algorithm A, keeping x6 and x7
>   
> ans <- cov.sel(T = datc$T, Y = datc$y, X = datc[,1:8], type="dr",
+                alpha = 0.1, alg = 1, scope=c("x6","x7"))

 dat$T ~ dat$x1 + dat$x2 + dat$x3 + dat$x4 + dat$x5 + 
     dat$x6 + dat$x7 + dat$x8
           Statistic      P.value
- dat$x5  0.09105982 7.284021e-01
- dat$x8  3.50028731 3.132293e-02
- dat$x1 25.80335207 5.051445e-09
- dat$x4 38.25479020 1.099454e-12
- dat$x2 43.38758663 3.452794e-14
- dat$x3 66.65306132 0.000000e+00

 dat$T ~ dat$x1 + dat$x2 + dat$x3 + dat$x4 + dat$x6 + 
     dat$x7 + dat$x8
         Statistic      P.value
- dat$x8  3.501491 3.130406e-02
- dat$x1 25.822950 4.995098e-09
- dat$x4 38.226052 1.124434e-12
- dat$x2 43.296547 3.685940e-14
- dat$x3 66.632932 0.000000e+00

Stopping Criterion Met

 data.0$Y ~ data.0$x1 + data.0$x2 + data.0$x3 + data.0$x4 + 
     data.0$x6 + data.0$x7 + data.0$x8
             Statistic   P.value
- data.0$x4   5.333877 0.5309639
- data.0$x3  10.298735 0.1193380
- data.0$x8  10.630418 0.1063049
- data.0$x1 110.065890 0.0000000
- data.0$x2 133.434654 0.0000000

 data.0$Y ~ data.0$x1 + data.0$x2 + data.0$x3 + data.0$x6 + 
     data.0$x7 + data.0$x8
             Statistic   P.value
- data.0$x8   4.950154 0.4577353
- data.0$x3   5.865198 0.3489299
- data.0$x1 110.606545 0.0000000
- data.0$x2 132.874828 0.0000000

 data.0$Y ~ data.0$x1 + data.0$x2 + data.0$x3 + data.0$x6 + 
     data.0$x7
            Statistic    P.value
- data.0$x3  10.49557 0.03542174
- data.0$x1 107.63187 0.00000000
- data.0$x2 136.41818 0.00000000

Stopping Criterion Met

 data.1$Y ~ data.1$x1 + data.1$x2 + data.1$x3 + data.1$x4 + 
     data.1$x6 + data.1$x7 + data.1$x8
             Statistic   P.value
- data.1$x3   2.853679 0.8554960
- data.1$x8   4.709078 0.6182049
- data.1$x4   5.134986 0.5624423
- data.1$x1  94.261903 0.0000000
- data.1$x2 127.716513 0.0000000

 data.1$Y ~ data.1$x1 + data.1$x2 + data.1$x4 + data.1$x6 + 
     data.1$x7 + data.1$x8
             Statistic   P.value
- data.1$x4   5.432994 0.3963586
- data.1$x8   6.636154 0.2717520
- data.1$x1  91.301401 0.0000000
- data.1$x2 133.035834 0.0000000

 data.1$Y ~ data.1$x1 + data.1$x2 + data.1$x6 + data.1$x7 + 
     data.1$x8
             Statistic   P.value
- data.1$x8   4.474502 0.3816548
- data.1$x1  94.970095 0.0000000
- data.1$x2 133.113032 0.0000000

 data.1$Y ~ data.1$x1 + data.1$x2 + data.1$x6 + data.1$x7
            Statistic P.value
- data.1$x1  94.90574       0
- data.1$x2 133.96681       0

Stopping Criterion Met
> 
> summary(ans)

 Original covariate vector: 
 x1 x2 x3 x4 x5 x6 x7 x8 
 
 Minimal subsets of the covariate vector: 
 Q.0  =  x1 x2 x3 x6 x7 
 Q.1  =  x1 x2 x6 x7 
 
 Removed variables: 
 Q.0comp  =  x4 x5 x8 
 Q.1comp  =  x3 x4 x5 x8 
 
 method  =  sir 
 
> 
> ##Algorithm B, method "save"
>   
> ans <- cov.sel(T = datc$T, Y = datc$y, X = datc[,1:10], type="dr",
+                alg = 2, method = "save", alpha = 0.3, na.action = "na.omit")

 data.0$Y ~ data.0$x1 + data.0$x2 + data.0$x3 + data.0$x4 + 
     data.0$x5 + data.0$x6 + data.0$x7 + data.0$x8 + 
     data.0$x9 + data.0$x10
             Statistic df(Nor) p.val(Nor) p.val(Gen)
- data.0$x4   9.542328      13 0.73086274 0.61612500
- data.0$x3  10.469487      13 0.65516559 0.55889387
- data.0$x9  13.196465      13 0.43275333 0.48656484
- data.0$x10 15.177922      13 0.29637760 0.27571589
- data.0$x1  23.801085      13 0.03300983 0.19531827
- data.0$x8  15.300474      13 0.28897221 0.16575346
- data.0$x7  17.485581      13 0.17804593 0.15091101
- data.0$x5  25.134036      13 0.02216506 0.03148484
- data.0$x6  21.065268      13 0.07164768 0.03092169
- data.0$x2  27.412510      13 0.01091815 0.00189721

 data.0$Y ~ data.0$x1 + data.0$x2 + data.0$x3 + data.0$x5 + 
     data.0$x6 + data.0$x7 + data.0$x8 + data.0$x9 + 
     data.0$x10
             Statistic df(Nor) p.val(Nor)  p.val(Gen)
- data.0$x3   9.326319      12 0.67483461 0.589751663
- data.0$x8  10.604810      12 0.56305504 0.416375788
- data.0$x9  15.701954      12 0.20527162 0.246544356
- data.0$x7  14.775660      12 0.25393485 0.225142938
- data.0$x1  23.669712      12 0.02254939 0.149767605
- data.0$x6  21.453902      12 0.04411598 0.017651022
- data.0$x10 23.921142      12 0.02084920 0.017177891
- data.0$x5  29.050174      12 0.00387296 0.006118409
- data.0$x2  25.131100      12 0.01421370 0.003114574

 data.0$Y ~ data.0$x1 + data.0$x2 + data.0$x5 + data.0$x6 + 
     data.0$x7 + data.0$x8 + data.0$x9 + data.0$x10
             Statistic df(Nor)  p.val(Nor)  p.val(Gen)
- data.0$x10  8.056677      11 0.708217927 0.674786220
- data.0$x8   9.320040      11 0.592377277 0.450582540
- data.0$x7  12.541739      11 0.324328305 0.282733188
- data.0$x9  15.387869      11 0.165419661 0.188928323
- data.0$x1  23.266545      11 0.016205870 0.121064696
- data.0$x6  20.897552      11 0.034448114 0.013664130
- data.0$x5  25.792411      11 0.006967484 0.013407819
- data.0$x2  22.113678      11 0.023507211 0.006483881

 data.0$Y ~ data.0$x1 + data.0$x2 + data.0$x5 + data.0$x6 + 
     data.0$x7 + data.0$x8 + data.0$x9
            Statistic df(Nor)  p.val(Nor)  p.val(Gen)
- data.0$x8  6.824829      10 0.741872229 0.627870116
- data.0$x9  9.376534      10 0.496780408 0.527727872
- data.0$x7 12.451667      10 0.255964326 0.219473738
- data.0$x1 23.362725      10 0.009484052 0.080912519
- data.0$x5 21.992898      10 0.015140823 0.026964097
- data.0$x2 20.310965      10 0.026444941 0.007955131
- data.0$x6 22.381889      10 0.013273024 0.004255483

 data.0$Y ~ data.0$x1 + data.0$x2 + data.0$x5 + data.0$x6 + 
     data.0$x7 + data.0$x9
            Statistic df(Nor) p.val(Nor)  p.val(Gen)
- data.0$x7  13.10585       9 0.15787477 0.112843794
- data.0$x1  21.40889       9 0.01095353 0.084193385
- data.0$x5  18.27122       9 0.03215413 0.051321303
- data.0$x9  18.27403       9 0.03212420 0.038398741
- data.0$x2  19.85192       9 0.01884861 0.004383159
- data.0$x6  21.28570       9 0.01144053 0.004142514

Stopping Criterion Met

 data.1$Y ~ data.1$x1 + data.1$x2 + data.1$x3 + data.1$x4 + 
     data.1$x5 + data.1$x6 + data.1$x7 + data.1$x8 + 
     data.1$x9 + data.1$x10
             Statistic df(Nor) p.val(Nor)  p.val(Gen)
- data.1$x10  7.797309      13 0.85654982 0.869762949
- data.1$x7   9.426885      13 0.73999314 0.853897676
- data.1$x8  10.746212      13 0.63206875 0.664025310
- data.1$x9  10.543716      13 0.64898235 0.644659717
- data.1$x4   9.841052      13 0.70687759 0.609122282
- data.1$x1  15.808997      13 0.25959798 0.395463070
- data.1$x2  19.384640      13 0.11165212 0.073571213
- data.1$x5  20.674619      13 0.07961861 0.059719425
- data.1$x6  21.821696      13 0.05818680 0.018230374
- data.1$x3  24.123385      13 0.03001389 0.006735747

 data.1$Y ~ data.1$x1 + data.1$x2 + data.1$x3 + data.1$x4 + 
     data.1$x5 + data.1$x6 + data.1$x7 + data.1$x8 + 
     data.1$x9
            Statistic df(Nor) p.val(Nor) p.val(Gen)
- data.1$x7  6.828572      12 0.86873114 0.93628626
- data.1$x8  6.826685      12 0.86885112 0.88026615
- data.1$x4 10.527076      12 0.56982240 0.45634670
- data.1$x9 12.300575      12 0.42185206 0.42472992
- data.1$x1 18.050010      12 0.11418047 0.20711570
- data.1$x5 19.261088      12 0.08241943 0.06596858
- data.1$x2 18.728734      12 0.09528857 0.06222762
- data.1$x3 18.178545      12 0.11037585 0.03780950
- data.1$x6 19.926045      12 0.06849793 0.02409641

 data.1$Y ~ data.1$x1 + data.1$x2 + data.1$x3 + data.1$x4 + 
     data.1$x5 + data.1$x6 + data.1$x8 + data.1$x9
            Statistic df(Nor) p.val(Nor) p.val(Gen)
- data.1$x9  10.71689      11 0.46727734 0.45244190
- data.1$x8  11.89614      11 0.37150042 0.28922935
- data.1$x4  12.03080      11 0.36133255 0.25979315
- data.1$x3  11.55762      11 0.39779634 0.23089368
- data.1$x5  15.15086      11 0.17569434 0.17106651
- data.1$x1  18.39095      11 0.07294143 0.14275654
- data.1$x6  15.23420      11 0.17202433 0.08290168
- data.1$x2  17.47154      11 0.09468076 0.05708977

 data.1$Y ~ data.1$x1 + data.1$x2 + data.1$x3 + data.1$x4 + 
     data.1$x5 + data.1$x6 + data.1$x8
            Statistic df(Nor) p.val(Nor)  p.val(Gen)
- data.1$x8  4.920705       9 0.84116510 0.806202290
- data.1$x4  7.654892       9 0.56926349 0.473899686
- data.1$x3  9.847840       9 0.36294339 0.220359048
- data.1$x5 15.191432       9 0.08581011 0.083091704
- data.1$x1 18.542778       9 0.02937374 0.062109988
- data.1$x2 16.007996       9 0.06671480 0.039454734
- data.1$x6 19.182283       9 0.02368682 0.006807779

 data.1$Y ~ data.1$x1 + data.1$x2 + data.1$x3 + data.1$x4 + 
     data.1$x5 + data.1$x6
            Statistic df(Nor) p.val(Nor) p.val(Gen)
- data.1$x4  9.925493       9 0.35655065 0.25706355
- data.1$x3  9.550890       9 0.38804993 0.25247465
- data.1$x1 15.890966       9 0.06919372 0.12529873
- data.1$x5 15.575323       9 0.07629725 0.07573507
- data.1$x2 15.560290       9 0.07665131 0.04746299
- data.1$x6 15.367451       9 0.08132472 0.03291357

Stopping Criterion Met

 dat$T ~ dat$x1 + dat$x2 + dat$x5 + dat$x6 + dat$x7 + 
     dat$x9
         Statistic df(Nor) p.val(Nor) p.val(Gen)
- dat$x5 0.2640375       1 0.60735978 0.61384136
- dat$x6 0.2283436       1 0.63275475 0.61142751
- dat$x1 1.0417311       1 0.30741920 0.36409583
- dat$x7 2.1708061       1 0.14065249 0.13520779
- dat$x2 4.0714770       1 0.04361312 0.03329171
- dat$x9 5.1086421       1 0.02380695 0.02705584

 dat$T ~ dat$x1 + dat$x2 + dat$x6 + dat$x7 + dat$x9
         Statistic df(Nor) p.val(Nor) p.val(Gen)
- dat$x6 0.1688472       1 0.68113836 0.66296097
- dat$x1 1.0410029       1 0.30758834 0.36418715
- dat$x7 2.2333714       1 0.13505886 0.13235782
- dat$x2 3.7610114       1 0.05246084 0.04143210
- dat$x9 5.2175459       1 0.02236008 0.02597992

 dat$T ~ dat$x1 + dat$x2 + dat$x7 + dat$x9
         Statistic df(Nor) p.val(Nor) p.val(Gen)
- dat$x1  1.025918       1 0.31111931 0.36824650
- dat$x7  2.225454       1 0.13575274 0.13192426
- dat$x2  3.791588       1 0.05151077 0.04086108
- dat$x9  5.459828       1 0.01945856 0.02267723

 dat$T ~ dat$x2 + dat$x7 + dat$x9
         Statistic df(Nor) p.val(Nor) p.val(Gen)
- dat$x7  2.245360       1 0.13401570 0.12989414
- dat$x2  3.782136       1 0.05180249 0.04113141
- dat$x9  5.392333       1 0.02022541 0.02371273

Stopping Criterion Met

 dat$T ~ dat$x1 + dat$x2 + dat$x3 + dat$x4 + dat$x5 + 
     dat$x6
         Statistic df(Nor) p.val(Nor) p.val(Gen)
- dat$x5 0.2403543       1 0.62395031 0.63234208
- dat$x6 0.2719172       1 0.60204860 0.57889311
- dat$x1 0.9790502       1 0.32243340 0.37751561
- dat$x4 1.6696486       1 0.19630561 0.16996999
- dat$x3 2.7365639       1 0.09807545 0.07010624
- dat$x2 4.1366325       1 0.04196457 0.03305899

 dat$T ~ dat$x1 + dat$x2 + dat$x3 + dat$x4 + dat$x6
         Statistic df(Nor) p.val(Nor) p.val(Gen)
- dat$x6 0.2095152       1 0.64714762 0.62675839
- dat$x1 0.9770909       1 0.32291808 0.37791627
- dat$x4 1.6704861       1 0.19619343 0.16981818
- dat$x3 2.7497451       1 0.09726993 0.06958060
- dat$x2 3.8205873       1 0.05062649 0.04125611

 dat$T ~ dat$x1 + dat$x2 + dat$x3 + dat$x4
         Statistic df(Nor) p.val(Nor) p.val(Gen)
- dat$x1 0.9626731       1 0.32651432 0.38196138
- dat$x4 1.6576223       1 0.19792466 0.17127860
- dat$x3 2.7358629       1 0.09811849 0.07009693
- dat$x2 3.8635119       1 0.04934693 0.04040920

 dat$T ~ dat$x2 + dat$x3 + dat$x4
         Statistic df(Nor) p.val(Nor) p.val(Gen)
- dat$x4  1.694675       1 0.19298586 0.16745579
- dat$x3  2.829606       1 0.09254105 0.06579833
- dat$x2  3.853221       1 0.04965055 0.04068917

Stopping Criterion Met
> 
> ## Kernel-based smoothing, both categorical and continuous covariates
> 
> data(datfc)
> ##The example below with default setting takes about 9 minutes to run.
> ## ans <- cov.sel(T = datfc$T, Y = datfc$y, X = datfc[,1:8], type="np",
> ##               alpha = 0.1, alg = 3, scope=NULL, thru=0.5, thro=0.25, thrc=100)
> 
> ## For illustration purposes we run Algorithm A using only the first 100 observations 
> ##and x1, x2, x3, x4 in datfc
> ans <- cov.sel(T = datfc$T[1:100], Y = datfc$y[1:100], X = datfc[1:100,1:4], 
+       type="np",alpha = 0.1, alg = 1, scope=NULL, thru=0.5, thro=0.25, thrc=100)
 Multistart 1 of 4 | Multistart 1 of 4 | Multistart 1 of 4 | Multistart 1 of 4 / Multistart 1 of 4 - Multistart 1 of 4  Multistart 1 of 4 | Multistart 1 of 4 | Multistart 1 of 4 / Multistart 2 of 4 | Multistart 2 of 4 | Multistart 2 of 4 / Multistart 2 of 4 - Multistart 2 of 4  Multistart 2 of 4 | Multistart 2 of 4 | Multistart 3 of 4 | Multistart 3 of 4 | Multistart 3 of 4 / Multistart 3 of 4 - Multistart 3 of 4  Multistart 3 of 4 | Multistart 3 of 4 | Multistart 3 of 4 / Multistart 3 of 4 - Multistart 4 of 4 | Multistart 4 of 4 | Multistart 4 of 4 / Multistart 4 of 4 - Multistart 4 of 4  Multistart 4 of 4 | Multistart 4 of 4 |                      Multistart 1 of 4 | Multistart 1 of 4 | Multistart 1 of 4 | Multistart 1 of 4 / Multistart 1 of 4 - Multistart 1 of 4 | Multistart 1 of 4 | Multistart 2 of 4 | Multistart 2 of 4 | Multistart 2 of 4 / Multistart 2 of 4 - Multistart 2 of 4  Multistart 2 of 4 | Multistart 2 of 4 | Multistart 3 of 4 | Multistart 3 of 4 | Multistart 3 of 4 / Multistart 3 of 4 - Multistart 3 of 4  Multistart 3 of 4 | Multistart 3 of 4 | Multistart 4 of 4 | Multistart 4 of 4 | Multistart 4 of 4 / Multistart 4 of 4 - Multistart 4 of 4  Multistart 4 of 4 | Multistart 4 of 4 | Multistart 4 of 4 |                      Multistart 1 of 4 | Multistart 1 of 4 | Multistart 1 of 4 | Multistart 1 of 4 / Multistart 1 of 4 - Multistart 1 of 4  Multistart 1 of 4 | Multistart 1 of 4 / Multistart 1 of 4 | Multistart 1 of 4 | Multistart 1 of 4 / Multistart 1 of 4 - Multistart 2 of 4 | Multistart 2 of 4 | Multistart 2 of 4 / Multistart 2 of 4 - Multistart 2 of 4  Multistart 2 of 4 | Multistart 2 of 4 | Multistart 2 of 4 | Multistart 3 of 4 | Multistart 3 of 4 | Multistart 3 of 4 / Multistart 3 of 4 - Multistart 3 of 4  Multistart 3 of 4 | Multistart 3 of 4 | Multistart 3 of 4 | Multistart 4 of 4 | Multistart 4 of 4 | Multistart 4 of 4 / Multistart 4 of 4 - Multistart 4 of 4  Multistart 4 of 4 | Multistart 4 of 4 | Multistart 4 of 4 | Multistart 4 of 4 / Multistart 4 of 4 -                     > 
> ##The example below running Algorithm A, keeping x6 and x7 with regtype="ll"
> ##takes about 7 minutes to run.
> ##ans <- cov.sel(T = datfc$T, Y = datfc$y, X = datfc[,1:8], type="np",
> ##               alpha = 0.1, alg = 3, scope=c("x6","x7"), thru=0.5, thro=0.25, 
> ##               thrc=100, regtype="ll")
> 
> 
> 
> 
> 
> 
> dev.off()
null device 
          1 
>