R: Fitting the Generalized and Variable Persistence Models
GPvam
R Documentation
Fitting the Generalized and Variable Persistence Models
Description
GPvam is used to fit the value-added model developed by Mariano et al. (2010) via ML estimation using an EM algorithm (Karl et al. 2013). Also provides the ability to fit the
variable persistence model (Lockwood et al. 2007).
a data frame that contains at least a column "y" containing the student scores, a
column "student" containing unique student ID's, a column "teacher" containing the
teacher ID's, and a column "year" which contains the year (or semester, etc.) of the time period. The "y" and "year" variables needs to be numeric. If other variables are to be included
as fixed effects, they should also be included in vam_data. See 'Note' for further discussion.
fixed_effects
an object of class formula describing the structure of the fixed effects. Categorical variables
should be wrapped in an as.factor statement.
student.side
a character. Choices are "G" or "R". See section 'Details'.
persistence
a character. Choices are "GP", "rGP", "VP", "CP", or "ZP". Only "GP" is currently
compatible with student.side="G". See section 'Details'.
max.iter.EM
the maximum number of EM iterations
tol1
convergence tolerance for EM algorithm. The convergence criterion is specified under 'Details'.
hessian
logical indicating whether the Hessian of the variance parameters (and persistence parameters for persistence="VP") should be calculated after convergence of the EM algorithm. Standard errors for the fixed and EBLUPs are calculated by default.
hes.method
a character string indicating the method of numerical differentiation used
to calculate the Hessian of the variance parameters. Options are "simple" or "richardson".
verbose
logical. If TRUE, model information will be printed at each iteration.
Details
The design for the random teacher effects according to the generalized persistence model
of Mariano et al. (2010) is built into the function. The model includes correlated current- and
future-year effects for each teacher. By setting student.side="R", the intra-student correlation is
modeled via an unstructured, block-diagonal error covariance matrix, as specified by Mariano et al. (2010). Setting
student.side="G" keeps the same teacher structure, but models intra-student correlation via
random student effects. This is similar to the model used by McCaffrey and Lockwood (2011), and is appropriate when the testing
scale is the same across years. In this case, the error covariance matrix is diagonal, although a separate variance is calcualted for each year.
From a computational perspective, the model estimating the R-side student effects has better scalability properties, although the G-side function is faster (Karl et al. 2012).
The persistence option determines the type of persistence effects that are modeled. The generalized persistence model ("GP") is described above. When student.side="R", other models for teacher persistence are available. The reduced GP model ("rGP", Karl et al. 2012) combines each teacher's future year effects from the GP model into a single effect. The variable persistence model ("VP") assumes that teacher effects in future years are multiples of their effect in the current year (Lockwood et al. 2007). The multipliers in the VP model are called persistence parameters, and are estimated. By contrast, the complete ("CP") and zero ("ZP") persistence models fix the persistence parameters at 1 and 0, respectively (Lockwood et al. 2007).
Convergence is declared when (l_k-l_{k-1})/l_k < 1E-07, where l_k is the log-likelihood
at iteration k.
The model is estimated via an EM algorithm. For details, see Karl et al. (2012). The model was estimated through Bayesian computation in Mariano et al. (2010).
Note: When student.side="R" is selected, the first few iterations of the EM algorithm will take longer than subsequent iterations. This is a result of the hybrid gradient-ascent/Newton-Raphson method used in the M-step for the R matrix in the first two iterations (Karl et al. 2012).
Program run time and memory requirements:
The data file GPvam.benchmark that is included with the package contains runtime and peak memory requirements for different persistence settings, using simulated data sets with different values for number of years, number of teachers per year, and number of students per teacher. These have been multiplied to show the total number of teachers in the data set, as well as the total number of students. With student.side="R", the persistence="GP" model is most sensitive to increases in the size of the data set. With student.side="G", the memory requirements increase exponentially with the number of students and teachers, and that model should not be considered scalable to extremely large data sets.
All of these benchmarks were performed with Hessian=TRUE. Calculation of the Hessian accounts for anywhere from 20% to 75% of those run times. Unless the standard errors of the variance components are needed, leaving Hessian=FALSE will lead to a faster run time with smaller memory requirements.
Value
GPvam returns an object of class GPvam
An object of class GPvam is a list containing the following components:
loglik
the maximized log-likelihood at convergence of the EM algorithm
teach.effects
a data frame containing the predicted teacher effects and standard errors
parameters
a matrix of estimated model parameters and standard errors
Hessian
if requested, the Hessian of the variance parameters
R_i
(only when student_side is set to 'R') a matrix containing the error covariance matrix of a student
teach.cov
a list containing the unique blocks of the covariance matrix of teacher effects
mresid
a vector of the raw marginal residuals
cresid
a vector of the raw conditional residuals
sresid
a vector of the scaled conditional residuals
yhat
a vector of the predicted values
The function summary provides a summary of the results. This includes the estimated model parameters and standard errors, along with the correlation matrices corresponding to the estimated correlation matrices. Summary information about scaled and raw residuals is reported.
Note
The model assumes that each teacher teaches only one year. If, for example, a teacher teaches in years 1 and 2, his/her first year performance is modeled independently of the second year performance.
To keep these effects seperate, the progam appends "(year i)" to each teacher name, where i is the year in which the teacher taught.
The fixed_effects argument of GPvam utilizes the functionality of R's formula class. In the statement fixed_effects=formula(~as.factor(year)+cont_var+0)), as.factor(year) identifies year as a categorical variable. +0 indicates that no intercept is to be fitted, and +cont_var indicates that a seperate effect is to be fitted for the continuous variable "cont_var." An interaction between "year" and "cont_var" could be specified by ~as.factor(year)*cont_var+0, or equivalently, ~as.factor(year)+cont_var+as.factor(year):cont_var+0. See formula for more details.
When applied to an object of class GPvam, plot.GPvam returns a caterpillar plot for each effect, as well as residual plots.
Karl, A., Yang, Y. and Lohr, S. (2013) Efficient Maximum Likelihood Estimation of Multiple Membership Linear Mixed Models, with an Application to Educational Value-Added Assessments Computational Statistics & Data Analysis59, 13–27.
Karl, A., Yang, Y. and Lohr, S. (2014) Computation of Maximum Likelihood Estimates for Multiresponse Generalized Linear Mixed Models with Non-nested, Correlated Random Effects Computational Statistics & Data Analysis73, 146–162.
Karl, A., Yang, Y. and Lohr, S. (2014) A Correlated Random Effects Model for Nonignorable Missing Data in Value-Added Assessment of Teacher Effects Journal of Educational and Behavioral Statistics38, 577–603.
Lockwood, J., McCaffrey, D., Mariano, L., Setodji, C. (2007) Bayesian Methods for Scalable Multivariate Value-Added Assesment. Journal of Educational and Behavioral Statistics32, 125–150.
Mariano, L., McCaffrey, D. and Lockwood, J. (2010) A Model for Teacher Effects From Longitudinal Data Without Assuming Vertical Scaling. Journal of Educational and Behavioral Statistics35,
253–279.
McCaffrey, D. and Lockwood, J. (2011) Missing Data in Value-Added Modeling of Teacher Effects," Annals of Applied Statistics5, 773–797
See Also
plot.GPvam,
summary.GPvam,
vam_data
Examples
data(vam_data)
GPvam(vam_data,student.side="R",persistence="VP",
fixed_effects=formula(~as.factor(year)+cont_var+0),verbose=TRUE,max.iter.EM=1)
result <- GPvam(vam_data,student.side="R",persistence="VP",
fixed_effects=formula(~as.factor(year)+cont_var+0),verbose=TRUE)
summary(result)
plot(result)