Optimizing Detection of True Within-Person Effects for Intensive Measurement Designs: A Comparison of Multilevel SEM and Unit-Weighted Scale Scores

Jonathan Rush, Philippe Rast, Scott M. Hofer

Research output: Contribution to journalArticle

Abstract

Intensive repeated measurement designs are frequently used to investigate within-person variation over relatively brief intervals of time. The majority of research utilizing these designs relies on unit-weighted scale scores, which assume that the constructs are measured without error. An alternative approach makes use of multilevel structural equation models (MSEM), which permit the specification of latent variables at both within-person and between-person levels. These models disaggregate measurement error from systematic variance, which should result in less biased within-person estimates and larger effect sizes. Differences in power, precision, and bias between multilevel unit-weighted models and MSEMs were compared through a series of Monte Carlo simulations. Results based on simulated data revealed that precision was consistently poorer in the MSEMs than the unit-weighted models, particularly when reliability was low. However, the degree of bias was considerably greater in the unit-weighted model than the latent variable model. Although the unit-weighted model consistently underestimated the effect of a covariate, it generally had similar power relative to the MSEM model due to the greater precision. Considerations for scale development and the impact of within-person reliability are highlighted.

Original languageEnglish (US)
JournalBehavior Research Methods
DOIs
StateAccepted/In press - Jan 1 2020

Keywords

  • Multilevel modeling
  • composite scores
  • multilevel structural equation modeling
  • power
  • within-person effects

ASJC Scopus subject areas

  • Experimental and Cognitive Psychology
  • Developmental and Educational Psychology
  • Arts and Humanities (miscellaneous)
  • Psychology (miscellaneous)
  • Psychology(all)

Fingerprint Dive into the research topics of 'Optimizing Detection of True Within-Person Effects for Intensive Measurement Designs: A Comparison of Multilevel SEM and Unit-Weighted Scale Scores'. Together they form a unique fingerprint.

  • Cite this