TY - JOUR
T1 - Optimizing Detection of True Within-Person Effects for Intensive Measurement Designs
T2 - A Comparison of Multilevel SEM and Unit-Weighted Scale Scores
AU - Rush, Jonathan
AU - Rast, Philippe
AU - Hofer, Scott M.
N1 - Funding Information:
Research reported in this manuscript was supported by the National Institute on Aging of the National Institutes of Health, Grant R01AG050720 and P01AG043362. Jonathan Rush was supported by a Joseph Armand Bombardier Doctoral Scholarship from the Social Sciences and Humanities Research Council of Canada.
Publisher Copyright:
© 2020, The Psychonomic Society, Inc.
Copyright:
Copyright 2020 Elsevier B.V., All rights reserved.
PY - 2020/10/1
Y1 - 2020/10/1
N2 - Intensive repeated measurement designs are frequently used to investigate within-person variation over relatively brief intervals of time. The majority of research utilizing these designs relies on unit-weighted scale scores, which assume that the constructs are measured without error. An alternative approach makes use of multilevel structural equation models (MSEM), which permit the specification of latent variables at both within-person and between-person levels. These models disaggregate measurement error from systematic variance, which should result in less biased within-person estimates and larger effect sizes. Differences in power, precision, and bias between multilevel unit-weighted models and MSEMs were compared through a series of Monte Carlo simulations. Results based on simulated data revealed that precision was consistently poorer in the MSEMs than the unit-weighted models, particularly when reliability was low. However, the degree of bias was considerably greater in the unit-weighted model than the latent variable model. Although the unit-weighted model consistently underestimated the effect of a covariate, it generally had similar power relative to the MSEM model due to the greater precision. Considerations for scale development and the impact of within-person reliability are highlighted.
AB - Intensive repeated measurement designs are frequently used to investigate within-person variation over relatively brief intervals of time. The majority of research utilizing these designs relies on unit-weighted scale scores, which assume that the constructs are measured without error. An alternative approach makes use of multilevel structural equation models (MSEM), which permit the specification of latent variables at both within-person and between-person levels. These models disaggregate measurement error from systematic variance, which should result in less biased within-person estimates and larger effect sizes. Differences in power, precision, and bias between multilevel unit-weighted models and MSEMs were compared through a series of Monte Carlo simulations. Results based on simulated data revealed that precision was consistently poorer in the MSEMs than the unit-weighted models, particularly when reliability was low. However, the degree of bias was considerably greater in the unit-weighted model than the latent variable model. Although the unit-weighted model consistently underestimated the effect of a covariate, it generally had similar power relative to the MSEM model due to the greater precision. Considerations for scale development and the impact of within-person reliability are highlighted.
KW - Multilevel modeling
KW - composite scores
KW - multilevel structural equation modeling
KW - power
KW - within-person effects
UR - http://www.scopus.com/inward/record.url?scp=85079815720&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85079815720&partnerID=8YFLogxK
U2 - 10.3758/s13428-020-01369-5
DO - 10.3758/s13428-020-01369-5
M3 - Article
C2 - 32072568
AN - SCOPUS:85079815720
VL - 52
SP - 1883
EP - 1892
JO - Behavior Research Methods
JF - Behavior Research Methods
SN - 1554-351X
IS - 5
ER -