The function tam.modelfit computes several model fit statistics.
It includes the Q3 statistic (Yen, 1984) and an
adjusted variant of it (see Details). Effect sizes of model fit
(MADaQ3, MADRESIDCOV,
SRMR) are also available.
The function IRT.modelfit is a wrapper to tam.modelfit,
but allows convenient model comparisons by using the
CDM::IRT.compareModels function.
The tam.modelfit function can also be used for fitted
models outside the TAM package by applying
tam.modelfit.IRT or tam.modelfit.args.
The function tam.Q3 computes the Q_3 statistic based on
weighted likelihood estimates (see tam.wle).
Usage
tam.modelfit(tamobj, progress = TRUE)
## S3 method for class 'tam.modelfit'
summary(object,...)
## S3 method for class 'tam.mml'
IRT.modelfit(object, ...)
## S3 method for class 'tam.mml.3pl'
IRT.modelfit(object, ...)
## S3 method for class 'tamaan'
IRT.modelfit(object, ...)
## S3 method for class 'IRT.modelfit.tam.mml'
summary(object, ...)
## S3 method for class 'IRT.modelfit.tam.mml.3pl'
summary(object, ...)
## S3 method for class 'IRT.modelfit.tamaan'
summary(object, ...)
tam.modelfit.IRT( object , progress=TRUE )
tam.modelfit.args( resp , probs , theta , post , progress=TRUE )
tam.Q3(tamobj , ... )
## S3 method for class 'tam.Q3'
summary(object,...)
Arguments
tamobj
Object of class tam
progress
An optional logical indicating whether progress should
be displayed
object
Object of class tam.modelfit (for summary)
or objects for which IRT.data, IRT.irfprob
and IRT.posterior have been defined (for tam.modelfit.IRT).
resp
Dataset with item responses
probs
Array with item response functions evaluated at theta
theta
Matrix with used old{θ} grid
post
Individual posterior distribution
...
Further arguments to be passed
Details
For each item i and each person n, residuals
e_{ni}=X_{ni}-E(X_{ni}) are computed. The expected
value E(X_{ni}) is obtained by integrating the individual
posterior distribution.
The Q3 statistic of item pairs i and j is defined as the
correlation Q3_{ij} = Cor( e_{ni} , e_{nj} ). The residuals in
tam.modelfit are
calculated by integrating values of the individual posterior distribution.
Residuals in tam.Q3 are calculated by using weighted likelihood
estimates (WLEs) from tam.wle.
It is known that under local independence, the expected value of Q_3
is slightly smaller than zero. Therefore,
an adjusted Q3 statistic (aQ3; aQ3_{ij})
is computed by subtracting the average of all Q3 statistics from
Q3. To control for multiple testing,
a p value adjustment by the method of
Holm (p.holm) is employed (see Chen, de la Torre & Zhang, 2013).
An effect size of model fit (MADaQ3) is defined as
the average of absolute values of aQ3 statistics. An equivalent
statistic based on the Q_3 statistic is similar to the
standardized generalized dimensionality discrepancy measure (SGDDM; Levy,
Xu, Yel & Svetina, 2015).
The SRMSR (standardized root mean square root of squared residuals,
Maydeu-Olivaras, 2013) is based on comparing residual correlations
of item pairs
This statistic is just multiplied by 100 in the output of this function.
Value
A list with following entries
stat.MADaQ3
Global fit statistic MADaQ3 and
global model test with p value obtained
by Holm adjustment
chi2.stat
Data frame with chi square tests of condional independence
for every item pair (Chen & Thissen, 1997)
fitstat
Model fit statistics 100 cdot MADRESIDCOV,
SRMR and SRMSR
modelfit.test
Test statistic of global fit based on multiple
testing correction of χ^2 statistics
stat.itempair
Q3 and adjusted Q3 statistic for all item pairs
residuals
Residuals
Q3.matr
Matrix of Q_3 statistics
aQ3.matr
Matrix of adjusted Q_3 statistics
Q3_summary
Summary of Q_3 statistics
N_itempair
Sample size for each item pair
References
Chen, J., de la Torre, J., & Zhang, Z. (2013).
Relative and absolute fit evaluation in cognitive diagnosis modeling.
Journal of Educational Measurement, 50, 123-140.
Chen, W., & Thissen, D. (1997). Local dependence indexes for item pairs
using item response theory. Journal of Educational and Behavioral Statistics,
22, 265-289.
Levy, R., Xu, Y., Yel, N., & Svetina, D. (2015). A standardized
generalized dimensionality discrepancy measure and a standardized model-based
covariance for dimensionality assessment for multidimensional models.
Journal of Educational Measurement, 52(2), 144–158.
Maydeu-Olivares, A. (2013). Goodness-of-fit assessment of item response
theory models (with discussion).
Measurement: Interdisciplinary Research and Perspectives,
11, 71-137.
McDonald, R. P., & Mok, M. M.-C. (1995). Goodness of fit in item response models.
Multivariate Behavioral Research, 30, 23-40.
Yen, W. M. (1984). Effects of local item dependence on the fit and equating
performance of the three-parameter logistic model.
Applied Psychological Measurement, 8, 125-145.
Examples
#############################################################################
# EXAMPLE 1: data.cqc01
#############################################################################
data(data.cqc01)
dat <- data.cqc01
#*****************************************************
#*** Model 1: Rasch model
mod1 <- tam.mml( dat )
# assess model fit
res1 <- tam.modelfit( tamobj = mod1 )
summary(res1)
# display item pairs with five largest adjusted Q3 statistics
res1$stat.itempair[1:5,c("item1","item2","aQ3","p","p.holm")]
## Not run:
# IRT.modelfit
fmod1 <- IRT.modelfit(mod1)
summary(fmod1)
#*****************************************************
#*** Model 2: 2PL model
mod2 <- tam.mml.2pl( dat )
# IRT.modelfit
fmod2 <- IRT.modelfit(mod2)
summary(fmod2)
# model comparison
IRT.compareModels(fmod1 , fmod2 )
#############################################################################
# SIMULATED EXAMPLE 2: Rasch model
#############################################################################
set.seed(8766)
N <- 1000 # number of persons
I <- 20 # number of items
# simulate responses
library(sirt)
dat <- sirt::sim.raschtype( rnorm(N) , b=seq(-1.5,1.5,len=I) )
#*** estimation
mod1 <- tam.mml( dat )
summary(dat)
#*** model fit
res1 <- tam.modelfit( tamobj = mod1)
summary(res1)
#############################################################################
# EXAMPLE 3: Model fit data.gpcm | Partial credit model
#############################################################################
data(data.gpcm)
dat <- data.gpcm
# estimate partial credit model
mod1 <- tam.mml( dat)
summary(mod1)
# assess model fit
tmod1 <- tam.modelfit( mod1 )
summary(tmod1)
#############################################################################
# EXAMPLE 4: data.read | Comparison Q3 statistic
#############################################################################
library(sirt)
data(data.read, package="sirt")
dat <- data.read
#**** Model 1: 1PL model
mod1 <- tam.mml( dat )
summary(mod1)
#**** Model 2: 2PL model
mod2 <- tam.mml.2pl( dat )
summary(mod2)
#**** assess model fits
# Q3 based on posterior
fmod1 <- tam.modelfit(mod1)
fmod2 <- tam.modelfit(mod2)
# Q3 based on WLEs
q3_mod1 <- tam.Q3(mod1)
q3_mod2 <- tam.Q3(mod2)
summary(fmod1)
summary(fmod2)
summary(q3_mod1)
summary(q3_mod2)
## End(Not run)