Last data update: 2014.03.03

 R: IQ-Learning: First-Stage Regression of Estimated Second-Stage...
iqLearnFSMR Documentation

IQ-Learning: First-Stage Regression of Estimated Second-Stage Main Effects

Description

Estimate an optimal dynamic treatment regime using the Interactive Q-learning (IQ-learning) algorithm when the data has been collected from a two-stage randomized trial with binary treatments coded as {-1,1}. iqLearnFSM implements the first-stage regression of the estimated second-stage main effects (IQ2).

Usage

 iqLearnFSM(..., moMain, moCont, data, response, txName, 
                   iter = 0L, suppress = FALSE)

Arguments

...

ignored

moMain

an object of class modelObj defining the regression model and procedures to be used for the main effects component of this regression step. The object must be created by a call to modelObj::buildModelObj(). The method chosen to obtain predictions must return the prediction on the scale of the response variable. See ?modelObj for details.

moCont

an object of class modelObj defining the regression model and procedures to be used for the contrast component of this regression step. The object must be created by a call to modelObj::buildModelObj(). The method chosen to obtain predictions must return the prediction on the scale of the response variable. See ?modelObj for details.

data

an object of class data.frame containing the covariates and treatment history. Treatment must be an integer (or able to be cast to an integer) and take values {-1,1}.

response

an object is of class iqLearnSS: the value object returned by a previous call to iqLearnSS().

txName

an object of class character: the column header of the first-stage treatment variable in data.

iter

an object of class integer.

>=1 if moMain and moCont are to be fitted separately, iter is the maximum number of iterations. Assume Y = Ymain + Ycont; the iterative algorithm is as follows: (1) hat(Ycont) = 0; (2) Ymain = Y - hat(Ycont); (3) fit Ymain ~ moMain; (4) set Ycont = Y - hat(Ymain); (5) fit Ycont ~ A*moCont; (6) Repeat steps (2) - (5) until convergence or a maximum of iter iterations.

<=0 if the components of the conditional expectation moMain and moCont will be combined and fit as a single object. Note that if iter <= 0, all non-model components of the moMain and moCont must be identical. By default, the choices in moMain are used.

suppress

an object of class logical. If TRUE, final screen prints are suppressed.

Details

There are standard regression analysis tools available for the object returned by this function. In general, these tools simply extend the methods defined by the regression function. If defined, coef() returns the model parameter estimates; plot() generates the standard x-y plots; residuals returns model residuals for the combined model; and summary returns summary information.

Other tools, such as fitted() for the lm regression function, can be accessed using fitObject(). fitObject() retrieves the standard value object returned by the regression method, which can be passed as input to other functions. See ?fitObject for details.

Value

Returns an object that inherits directly from class DynTxRegime.

Author(s)

Kristin A. Linn, Eric B. Laber, Leonard A. Stefanski, and Shannon T. Holloway <sthollow@ncsu.edu>

References

Laber, E. B., Linn, K. A., and Stefanski, L.A. (2014). Interactive Q-learning. Biometrika, in press.

See Also

iqLearnSS, iqLearnFSC iqLearnFSV

Examples

  ##########################################################
  # Load and process data set
  ##########################################################
    data(bmiData)

    #----------------------------------------------------#
    # Recast treatment variables to (-1,1) --- required.
    #----------------------------------------------------#
    bmiData$A1[which (bmiData$A1=="MR")] <- 1L
    bmiData$A1[which (bmiData$A1=="CD")] <- -1L
    bmiData$A2[which (bmiData$A2=="MR")] <- 1L
    bmiData$A2[which (bmiData$A2=="CD")] <- -1L
    bmiData$A1 <- as.integer(bmiData$A1)
    bmiData$A2 <- as.integer(bmiData$A2)

    #----------------------------------------------------#
    # define response y to be the negative 12 month
    # change in BMI from baseline
    #----------------------------------------------------#
    bmiData$y <- -100*(bmiData[,6] - bmiData[,4])/bmiData[,4]

  ##########################################################
  # Second-stage regression
  ##########################################################
    #----------------------------------------------------#
    # Create modelObj object for main effect component
    #----------------------------------------------------#
    moMain <- buildModelObj(model = ~ gender + parentBMI + month4BMI,
                            solver.method = 'lm')

    #----------------------------------------------------#
    # Create modelObj object for contrast component
    #----------------------------------------------------#
    moCont <- buildModelObj(model = ~ parentBMI + month4BMI,
                            solver.method = 'lm')

    iqSS <- iqLearnSS(moMain = moMain, 
                      moCont = moCont, 
                      data = bmiData,
                      response = bmiData$y, 
                      txName = "A2", 
                      iter = 0)

  ##########################################################
  # First-stage regression - Single Regression Step
  ##########################################################
    #----------------------------------------------------#
    # Create modelObj object for main effect component
    #----------------------------------------------------#
    moMain <- buildModelObj(model = ~ gender + race + parentBMI + baselineBMI,
                            solver.method = 'lm')

    #----------------------------------------------------#
    # Create modelObj object for contrast component
    #----------------------------------------------------#
    moCont <- buildModelObj(model = ~ parentBMI + month4BMI,
                            solver.method = 'lm')

    iqFSM <- iqLearnFSM(moMain = moMain, 
                        moCont = moCont, 
                        data = bmiData,
                        response = iqSS, 
                        txName = "A1", 
                        iter = 0)

    # Estimated Q-functions
    vals <- qFuncs(iqFSM)
    head(vals)

    # Residuals
    res <- residuals(iqFSM)
    head(res)

    # Model parameter estimates
    coef(iqFSM)

    # Summary information for regression analysis
    summary(iqFSM)

    # Standard lm plots
    plot(iqFSM)

    # Value object returned by lm
    fitObj <- fitObject(iqFSM)
    fitObj

    # All standard lm  methods can be applied to the elements of this list.
    summary(fitObj[[ "Combined" ]])
    coef(fitObj[[ "Combined" ]])
    head(residuals(fitObj[[ "Combined" ]]))
    head(fitted.values(fitObj[[ "Combined" ]]))
    plot(fitObj[[ "Combined"]])


  ##########################################################
  # First-stage regression - Iterative Regression Step
  ##########################################################
    iqFSM <- iqLearnFSM(moMain = moMain, 
                        moCont = moCont, 
                        data = bmiData,
                        response = iqSS, 
                        txName = "A1", 
                        iter = 100)

    # Estimated Q-functions
    vals <- qFuncs(iqFSM)
    head(vals)

    # Residuals
    res <- residuals(iqFSM)
    head(res)

    # Model parameter estimates
    coef(iqFSM)

    # Summary information for regression analysis
    summary(iqFSM)

    # Standard lm plots
    plot(iqFSM)

    # Value object returned by lm
    fitObj <- fitObject(iqFSM)
    fitObj

    # All standard lm  methods can be applied to the elements of this list.
    summary(fitObj[[ "MainEffect" ]])
    coef(fitObj[[ "MainEffect" ]])
    head(residuals(fitObj[[ "MainEffect" ]]))
    head(fitted.values(fitObj[[ "MainEffect" ]]))
    plot(fitObj[[ "MainEffect"]])

    summary(fitObj[[ "Contrast" ]])
    coef(fitObj[[ "Contrast" ]])
    head(residuals(fitObj[[ "Contrast" ]]))
    head(fitted.values(fitObj[[ "Contrast" ]]))
    plot(fitObj[[ "Contrast"]])

Results


R version 3.3.1 (2016-06-21) -- "Bug in Your Hair"
Copyright (C) 2016 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> library(DynTxRegime)
Loading required package: modelObj
> png(filename="/home/ddbj/snapshot/RGM3/R_CC/result/DynTxRegime/iqLearnFS.Rd_%03d_medium.png", width=480, height=480)
> ### Name: iqLearnFSM
> ### Title: IQ-Learning: First-Stage Regression of Estimated Second-Stage
> ###   Main Effects
> ### Aliases: iqLearnFSM
> 
> ### ** Examples
> 
>   ##########################################################
>   # Load and process data set
>   ##########################################################
>     data(bmiData)
> 
>     #----------------------------------------------------#
>     # Recast treatment variables to (-1,1) --- required.
>     #----------------------------------------------------#
>     bmiData$A1[which (bmiData$A1=="MR")] <- 1L
>     bmiData$A1[which (bmiData$A1=="CD")] <- -1L
>     bmiData$A2[which (bmiData$A2=="MR")] <- 1L
>     bmiData$A2[which (bmiData$A2=="CD")] <- -1L
>     bmiData$A1 <- as.integer(bmiData$A1)
>     bmiData$A2 <- as.integer(bmiData$A2)
> 
>     #----------------------------------------------------#
>     # define response y to be the negative 12 month
>     # change in BMI from baseline
>     #----------------------------------------------------#
>     bmiData$y <- -100*(bmiData[,6] - bmiData[,4])/bmiData[,4]
> 
>   ##########################################################
>   # Second-stage regression
>   ##########################################################
>     #----------------------------------------------------#
>     # Create modelObj object for main effect component
>     #----------------------------------------------------#
>     moMain <- buildModelObj(model = ~ gender + parentBMI + month4BMI,
+                             solver.method = 'lm')
> 
>     #----------------------------------------------------#
>     # Create modelObj object for contrast component
>     #----------------------------------------------------#
>     moCont <- buildModelObj(model = ~ parentBMI + month4BMI,
+                             solver.method = 'lm')
> 
>     iqSS <- iqLearnSS(moMain = moMain, 
+                       moCont = moCont, 
+                       data = bmiData,
+                       response = bmiData$y, 
+                       txName = "A2", 
+                       iter = 0)

 *** Combined Fit ***

Call:
lm(formula = YinternalY ~ gender + parentBMI + month4BMI + A2 + 
    parentBMI:A2 + month4BMI:A2, data = data)

Coefficients:
 (Intercept)        gender     parentBMI     month4BMI            A2  
    41.28845      -0.64891      -0.15509      -0.82067      -7.38709  
parentBMI:A2  month4BMI:A2  
     0.20223       0.02816  


Mean of Value Function:  7.646356 

> 
>   ##########################################################
>   # First-stage regression - Single Regression Step
>   ##########################################################
>     #----------------------------------------------------#
>     # Create modelObj object for main effect component
>     #----------------------------------------------------#
>     moMain <- buildModelObj(model = ~ gender + race + parentBMI + baselineBMI,
+                             solver.method = 'lm')
> 
>     #----------------------------------------------------#
>     # Create modelObj object for contrast component
>     #----------------------------------------------------#
>     moCont <- buildModelObj(model = ~ parentBMI + month4BMI,
+                             solver.method = 'lm')
> 
>     iqFSM <- iqLearnFSM(moMain = moMain, 
+                         moCont = moCont, 
+                         data = bmiData,
+                         response = iqSS, 
+                         txName = "A1", 
+                         iter = 0)

 *** Combined Fit ***

Call:
lm(formula = YinternalY ~ gender + race + parentBMI + baselineBMI + 
    A1 + parentBMI:A1 + A1:month4BMI, data = data)

Coefficients:
 (Intercept)        gender          race     parentBMI   baselineBMI  
    40.29177      -0.56198      -0.18294      -0.38722      -0.53338  
          A1  parentBMI:A1  A1:month4BMI  
     3.37120      -0.19882       0.08342  

> 
>     # Estimated Q-functions
>     vals <- qFuncs(iqFSM)
>     head(vals)
           -1         1
[1,] 8.813094  8.701766
[2,] 7.741409  8.554373
[3,] 8.133323  8.578383
[4,] 8.887424 10.123289
[5,] 9.799654 11.662326
[6,] 9.155772  9.593364
> 
>     # Residuals
>     res <- residuals(iqFSM)
>     head(res)
[1] -0.5142056 -1.6379011 -0.8836873  0.8000097 -0.9386964  0.8336625
> 
>     # Model parameter estimates
>     coef(iqFSM)
$Combined
 (Intercept)       gender         race    parentBMI  baselineBMI           A1 
 40.29176873  -0.56197536  -0.18293646  -0.38722292  -0.53338052   3.37120326 
parentBMI:A1 A1:month4BMI 
 -0.19881529   0.08341519 

> 
>     # Summary information for regression analysis
>     summary(iqFSM)
$Combined

Call:
lm(formula = YinternalY ~ gender + race + parentBMI + baselineBMI + 
    A1 + parentBMI:A1 + A1:month4BMI, data = data)

Residuals:
   Min     1Q Median     3Q    Max 
-4.361 -1.067  0.160  1.198  5.287 

Coefficients:
             Estimate Std. Error t value Pr(>|t|)    
(Intercept)  40.29177    1.23436  32.642  < 2e-16 ***
gender       -0.56198    0.24026  -2.339  0.02031 *  
race         -0.18294    0.24067  -0.760  0.44807    
parentBMI    -0.38722    0.02385 -16.237  < 2e-16 ***
baselineBMI  -0.53338    0.03516 -15.171  < 2e-16 ***
A1            3.37120    1.09616   3.075  0.00239 ** 
parentBMI:A1 -0.19882    0.02676  -7.429 3.01e-12 ***
A1:month4BMI  0.08342    0.03940   2.117  0.03547 *  
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 1.709 on 202 degrees of freedom
Multiple R-squared:  0.8245,	Adjusted R-squared:  0.8184 
F-statistic: 135.6 on 7 and 202 DF,  p-value: < 2.2e-16


> 
>     # Standard lm plots
>     plot(iqFSM)
> 
>     # Value object returned by lm
>     fitObj <- fitObject(iqFSM)
>     fitObj
$Combined

Call:
lm(formula = YinternalY ~ gender + race + parentBMI + baselineBMI + 
    A1 + parentBMI:A1 + A1:month4BMI, data = data)

Coefficients:
 (Intercept)        gender          race     parentBMI   baselineBMI  
    40.29177      -0.56198      -0.18294      -0.38722      -0.53338  
          A1  parentBMI:A1  A1:month4BMI  
     3.37120      -0.19882       0.08342  


> 
>     # All standard lm  methods can be applied to the elements of this list.
>     summary(fitObj[[ "Combined" ]])

Call:
lm(formula = YinternalY ~ gender + race + parentBMI + baselineBMI + 
    A1 + parentBMI:A1 + A1:month4BMI, data = data)

Residuals:
   Min     1Q Median     3Q    Max 
-4.361 -1.067  0.160  1.198  5.287 

Coefficients:
             Estimate Std. Error t value Pr(>|t|)    
(Intercept)  40.29177    1.23436  32.642  < 2e-16 ***
gender       -0.56198    0.24026  -2.339  0.02031 *  
race         -0.18294    0.24067  -0.760  0.44807    
parentBMI    -0.38722    0.02385 -16.237  < 2e-16 ***
baselineBMI  -0.53338    0.03516 -15.171  < 2e-16 ***
A1            3.37120    1.09616   3.075  0.00239 ** 
parentBMI:A1 -0.19882    0.02676  -7.429 3.01e-12 ***
A1:month4BMI  0.08342    0.03940   2.117  0.03547 *  
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 1.709 on 202 degrees of freedom
Multiple R-squared:  0.8245,	Adjusted R-squared:  0.8184 
F-statistic: 135.6 on 7 and 202 DF,  p-value: < 2.2e-16

>     coef(fitObj[[ "Combined" ]])
 (Intercept)       gender         race    parentBMI  baselineBMI           A1 
 40.29176873  -0.56197536  -0.18293646  -0.38722292  -0.53338052   3.37120326 
parentBMI:A1 A1:month4BMI 
 -0.19881529   0.08341519 
>     head(residuals(fitObj[[ "Combined" ]]))
         1          2          3          4          5          6 
-0.5142056 -1.6379011 -0.8836873  0.8000097 -0.9386964  0.8336625 
>     head(fitted.values(fitObj[[ "Combined" ]]))
       1        2        3        4        5        6 
8.813094 7.741409 8.578383 8.887424 9.799654 9.593364 
>     plot(fitObj[[ "Combined"]])
> 
> 
>   ##########################################################
>   # First-stage regression - Iterative Regression Step
>   ##########################################################
>     iqFSM <- iqLearnFSM(moMain = moMain, 
+                         moCont = moCont, 
+                         data = bmiData,
+                         response = iqSS, 
+                         txName = "A1", 
+                         iter = 100)

 *** MainEffect Fit *** 

Call:
lm(formula = YinternalY ~ gender + race + parentBMI + baselineBMI, 
    data = data)

Coefficients:
(Intercept)       gender         race    parentBMI  baselineBMI  
    40.2918      -0.5620      -0.1829      -0.3872      -0.5334  


 *** Contrast Fit *** 

Call:
lm(formula = YinternalY ~ A1 + A1:parentBMI + A1:month4BMI - 
    1, data = data)

Coefficients:
          A1  A1:parentBMI  A1:month4BMI  
     3.37120      -0.19882       0.08342  

> 
>     # Estimated Q-functions
>     vals <- qFuncs(iqFSM)
>     head(vals)
           -1         1
[1,] 8.813094  8.701766
[2,] 7.741409  8.554373
[3,] 8.133323  8.578383
[4,] 8.887424 10.123289
[5,] 9.799654 11.662326
[6,] 9.155772  9.593364
> 
>     # Residuals
>     res <- residuals(iqFSM)
>     head(res)
[1] -0.5142056 -1.6379011 -0.8836873  0.8000097 -0.9386964  0.8336625
> 
>     # Model parameter estimates
>     coef(iqFSM)
$MainEffect
(Intercept)      gender        race   parentBMI baselineBMI 
 40.2917687  -0.5619754  -0.1829365  -0.3872229  -0.5333805 

$Contrast
          A1 A1:parentBMI A1:month4BMI 
  3.37120324  -0.19881529   0.08341519 

> 
>     # Summary information for regression analysis
>     summary(iqFSM)
$MainEffect

Call:
lm(formula = YinternalY ~ gender + race + parentBMI + baselineBMI, 
    data = data)

Residuals:
   Min     1Q Median     3Q    Max 
-4.361 -1.067  0.160  1.198  5.287 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept) 40.29177    1.20960  33.310   <2e-16 ***
gender      -0.56198    0.23549  -2.386   0.0179 *  
race        -0.18294    0.23621  -0.774   0.4395    
parentBMI   -0.38722    0.02214 -17.489   <2e-16 ***
baselineBMI -0.53338    0.03315 -16.090   <2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 1.697 on 205 degrees of freedom
Multiple R-squared:  0.8188,	Adjusted R-squared:  0.8153 
F-statistic: 231.6 on 4 and 205 DF,  p-value: < 2.2e-16


$Contrast

Call:
lm(formula = YinternalY ~ A1 + A1:parentBMI + A1:month4BMI - 
    1, data = data)

Residuals:
   Min     1Q Median     3Q    Max 
-4.361 -1.067  0.160  1.198  5.287 

Coefficients:
             Estimate Std. Error t value Pr(>|t|)    
A1            3.37120    1.01409   3.324  0.00105 ** 
A1:parentBMI -0.19882    0.02600  -7.646 7.67e-13 ***
A1:month4BMI  0.08342    0.03623   2.303  0.02229 *  
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 1.688 on 207 degrees of freedom
Multiple R-squared:  0.2712,	Adjusted R-squared:  0.2606 
F-statistic: 25.67 on 3 and 207 DF,  p-value: 3.674e-14


> 
>     # Standard lm plots
>     plot(iqFSM)
> 
>     # Value object returned by lm
>     fitObj <- fitObject(iqFSM)
>     fitObj
$MainEffect

Call:
lm(formula = YinternalY ~ gender + race + parentBMI + baselineBMI, 
    data = data)

Coefficients:
(Intercept)       gender         race    parentBMI  baselineBMI  
    40.2918      -0.5620      -0.1829      -0.3872      -0.5334  


$Contrast

Call:
lm(formula = YinternalY ~ A1 + A1:parentBMI + A1:month4BMI - 
    1, data = data)

Coefficients:
          A1  A1:parentBMI  A1:month4BMI  
     3.37120      -0.19882       0.08342  


> 
>     # All standard lm  methods can be applied to the elements of this list.
>     summary(fitObj[[ "MainEffect" ]])

Call:
lm(formula = YinternalY ~ gender + race + parentBMI + baselineBMI, 
    data = data)

Residuals:
   Min     1Q Median     3Q    Max 
-4.361 -1.067  0.160  1.198  5.287 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept) 40.29177    1.20960  33.310   <2e-16 ***
gender      -0.56198    0.23549  -2.386   0.0179 *  
race        -0.18294    0.23621  -0.774   0.4395    
parentBMI   -0.38722    0.02214 -17.489   <2e-16 ***
baselineBMI -0.53338    0.03315 -16.090   <2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 1.697 on 205 degrees of freedom
Multiple R-squared:  0.8188,	Adjusted R-squared:  0.8153 
F-statistic: 231.6 on 4 and 205 DF,  p-value: < 2.2e-16

>     coef(fitObj[[ "MainEffect" ]])
(Intercept)      gender        race   parentBMI baselineBMI 
 40.2917687  -0.5619754  -0.1829365  -0.3872229  -0.5333805 
>     head(residuals(fitObj[[ "MainEffect" ]]))
         1          2          3          4          5          6 
-0.5142056 -1.6379011 -0.8836873  0.8000097 -0.9386964  0.8336625 
>     head(fitted.values(fitObj[[ "MainEffect" ]]))
        1         2         3         4         5         6 
 8.757430  8.147891  8.355853  9.505356 10.730990  9.374568 
>     plot(fitObj[[ "MainEffect"]])
> 
>     summary(fitObj[[ "Contrast" ]])

Call:
lm(formula = YinternalY ~ A1 + A1:parentBMI + A1:month4BMI - 
    1, data = data)

Residuals:
   Min     1Q Median     3Q    Max 
-4.361 -1.067  0.160  1.198  5.287 

Coefficients:
             Estimate Std. Error t value Pr(>|t|)    
A1            3.37120    1.01409   3.324  0.00105 ** 
A1:parentBMI -0.19882    0.02600  -7.646 7.67e-13 ***
A1:month4BMI  0.08342    0.03623   2.303  0.02229 *  
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 1.688 on 207 degrees of freedom
Multiple R-squared:  0.2712,	Adjusted R-squared:  0.2606 
F-statistic: 25.67 on 3 and 207 DF,  p-value: 3.674e-14

>     coef(fitObj[[ "Contrast" ]])
          A1 A1:parentBMI A1:month4BMI 
  3.37120324  -0.19881529   0.08341519 
>     head(residuals(fitObj[[ "Contrast" ]]))
         1          2          3          4          5          6 
-0.5142056 -1.6379011 -0.8836873  0.8000097 -0.9386964  0.8336625 
>     head(fitted.values(fitObj[[ "Contrast" ]]))
          1           2           3           4           5           6 
 0.05566437 -0.40648174  0.22253038 -0.61793258 -0.93133594  0.21879579 
>     plot(fitObj[[ "Contrast"]])
> 
> 
> 
> 
> 
> 
> dev.off()
null device 
          1 
>


Created & Maintained by Osamu Ogasawara (osamu.ogasawara@gmail.com) and IMSBIO