Automated Scoring of Tablet-Administered Expressive Language Tests

Robert Gale, Julie Bird, Yiyi Wang, Jan Van Santen, Emily Prud'hommeaux, Jill Dolata, Meysam Asgari

Research output: Contribution to journalArticlepeer-review

Abstract

Speech and language impairments are common pediatric conditions, with as many as 10% of children experiencing one or both at some point during development. Expressive language disorders in particular often go undiagnosed, underscoring the immediate need for assessments of expressive language that can be administered and scored reliably and objectively. In this paper, we present a set of highly accurate computational models for automatically scoring several common expressive language tasks. In our assessment framework, instructions and stimuli are presented to the child on a tablet computer, which records the child's responses in real time, while a clinician controls the pace and presentation of the tasks using a second tablet. The recorded responses for four distinct expressive language tasks (expressive vocabulary, word structure, recalling sentences, and formulated sentences) are then scored using traditional paper-and-pencil scoring and using machine learning methods relying on a deep neural network-based language representation model. All four tasks can be scored automatically from both clean and verbatim speech transcripts with very high accuracy at the item level (83−99%). In addition, these automated scores correlate strongly and significantly (ρ = 0.76–0.99, p < 0.001) with manual item-level, raw, and scaled scores. These results point to the utility and potential of automated computationally-driven methods of both administering and scoring expressive language tasks for pediatric developmental language evaluation.

Original languageEnglish (US)
Article number668401
JournalFrontiers in Psychology
Volume12
DOIs
StatePublished - Jul 22 2021

Keywords

  • assessment
  • automated scoring
  • expressive language
  • language disorders
  • neural language models
  • speech

ASJC Scopus subject areas

  • Psychology(all)

Fingerprint

Dive into the research topics of 'Automated Scoring of Tablet-Administered Expressive Language Tests'. Together they form a unique fingerprint.

Cite this