Automatic scoring of a nonword repetition test

Meysam Asgari, Jan Van Santen, Katina Papadakis

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In this study, we explore the feasibility of speech-based techniques to automatically evaluate a nonword repetition (NWR) test. NWR tests, a useful marker for detecting language impairment, require repetition of pronounceable nonwords, such as 'D OY F', presented aurally by an examiner or via a recording. Our proposed method leverages ASR techniques to first transcribe verbal responses. Second, it applies machine learning techniques to ASR output for predicting gold standard scores provided by speech and language pathologists. Our experimental results for a sample of 101 children (42 with autism spectrum disorders, or ASD; 18 with specific language impairment, or SLI; and 41 typically developed, or TD) show that the proposed approach is successful in predicting scores on this test, with averaged product-moment correlations of 0.74 and mean absolute error of 0.06 (on a observed score range from 0.34 to 0.97) between observed and predicted ratings.

Original languageEnglish (US)
Title of host publicationProceedings - 16th IEEE International Conference on Machine Learning and Applications, ICMLA 2017
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages304-308
Number of pages5
Volume2018-January
ISBN (Electronic)9781538614174
DOIs
StatePublished - Jan 16 2018
Event16th IEEE International Conference on Machine Learning and Applications, ICMLA 2017 - Cancun, Mexico
Duration: Dec 18 2017Dec 21 2017

Other

Other16th IEEE International Conference on Machine Learning and Applications, ICMLA 2017
CountryMexico
CityCancun
Period12/18/1712/21/17

Fingerprint

Learning systems

Keywords

  • Autism Spectrum Disorder
  • Automatic Scoring
  • Nonword stimuli repetition

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Science Applications

Cite this

Asgari, M., Van Santen, J., & Papadakis, K. (2018). Automatic scoring of a nonword repetition test. In Proceedings - 16th IEEE International Conference on Machine Learning and Applications, ICMLA 2017 (Vol. 2018-January, pp. 304-308). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICMLA.2017.0-143

Automatic scoring of a nonword repetition test. / Asgari, Meysam; Van Santen, Jan; Papadakis, Katina.

Proceedings - 16th IEEE International Conference on Machine Learning and Applications, ICMLA 2017. Vol. 2018-January Institute of Electrical and Electronics Engineers Inc., 2018. p. 304-308.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Asgari, M, Van Santen, J & Papadakis, K 2018, Automatic scoring of a nonword repetition test. in Proceedings - 16th IEEE International Conference on Machine Learning and Applications, ICMLA 2017. vol. 2018-January, Institute of Electrical and Electronics Engineers Inc., pp. 304-308, 16th IEEE International Conference on Machine Learning and Applications, ICMLA 2017, Cancun, Mexico, 12/18/17. https://doi.org/10.1109/ICMLA.2017.0-143
Asgari M, Van Santen J, Papadakis K. Automatic scoring of a nonword repetition test. In Proceedings - 16th IEEE International Conference on Machine Learning and Applications, ICMLA 2017. Vol. 2018-January. Institute of Electrical and Electronics Engineers Inc. 2018. p. 304-308 https://doi.org/10.1109/ICMLA.2017.0-143
Asgari, Meysam ; Van Santen, Jan ; Papadakis, Katina. / Automatic scoring of a nonword repetition test. Proceedings - 16th IEEE International Conference on Machine Learning and Applications, ICMLA 2017. Vol. 2018-January Institute of Electrical and Electronics Engineers Inc., 2018. pp. 304-308
@inproceedings{e5683ce1315f40df9853273d001ba0e2,
title = "Automatic scoring of a nonword repetition test",
abstract = "In this study, we explore the feasibility of speech-based techniques to automatically evaluate a nonword repetition (NWR) test. NWR tests, a useful marker for detecting language impairment, require repetition of pronounceable nonwords, such as 'D OY F', presented aurally by an examiner or via a recording. Our proposed method leverages ASR techniques to first transcribe verbal responses. Second, it applies machine learning techniques to ASR output for predicting gold standard scores provided by speech and language pathologists. Our experimental results for a sample of 101 children (42 with autism spectrum disorders, or ASD; 18 with specific language impairment, or SLI; and 41 typically developed, or TD) show that the proposed approach is successful in predicting scores on this test, with averaged product-moment correlations of 0.74 and mean absolute error of 0.06 (on a observed score range from 0.34 to 0.97) between observed and predicted ratings.",
keywords = "Autism Spectrum Disorder, Automatic Scoring, Nonword stimuli repetition",
author = "Meysam Asgari and {Van Santen}, Jan and Katina Papadakis",
year = "2018",
month = "1",
day = "16",
doi = "10.1109/ICMLA.2017.0-143",
language = "English (US)",
volume = "2018-January",
pages = "304--308",
booktitle = "Proceedings - 16th IEEE International Conference on Machine Learning and Applications, ICMLA 2017",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - GEN

T1 - Automatic scoring of a nonword repetition test

AU - Asgari, Meysam

AU - Van Santen, Jan

AU - Papadakis, Katina

PY - 2018/1/16

Y1 - 2018/1/16

N2 - In this study, we explore the feasibility of speech-based techniques to automatically evaluate a nonword repetition (NWR) test. NWR tests, a useful marker for detecting language impairment, require repetition of pronounceable nonwords, such as 'D OY F', presented aurally by an examiner or via a recording. Our proposed method leverages ASR techniques to first transcribe verbal responses. Second, it applies machine learning techniques to ASR output for predicting gold standard scores provided by speech and language pathologists. Our experimental results for a sample of 101 children (42 with autism spectrum disorders, or ASD; 18 with specific language impairment, or SLI; and 41 typically developed, or TD) show that the proposed approach is successful in predicting scores on this test, with averaged product-moment correlations of 0.74 and mean absolute error of 0.06 (on a observed score range from 0.34 to 0.97) between observed and predicted ratings.

AB - In this study, we explore the feasibility of speech-based techniques to automatically evaluate a nonword repetition (NWR) test. NWR tests, a useful marker for detecting language impairment, require repetition of pronounceable nonwords, such as 'D OY F', presented aurally by an examiner or via a recording. Our proposed method leverages ASR techniques to first transcribe verbal responses. Second, it applies machine learning techniques to ASR output for predicting gold standard scores provided by speech and language pathologists. Our experimental results for a sample of 101 children (42 with autism spectrum disorders, or ASD; 18 with specific language impairment, or SLI; and 41 typically developed, or TD) show that the proposed approach is successful in predicting scores on this test, with averaged product-moment correlations of 0.74 and mean absolute error of 0.06 (on a observed score range from 0.34 to 0.97) between observed and predicted ratings.

KW - Autism Spectrum Disorder

KW - Automatic Scoring

KW - Nonword stimuli repetition

UR - http://www.scopus.com/inward/record.url?scp=85048510807&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85048510807&partnerID=8YFLogxK

U2 - 10.1109/ICMLA.2017.0-143

DO - 10.1109/ICMLA.2017.0-143

M3 - Conference contribution

VL - 2018-January

SP - 304

EP - 308

BT - Proceedings - 16th IEEE International Conference on Machine Learning and Applications, ICMLA 2017

PB - Institute of Electrical and Electronics Engineers Inc.

ER -