TY - GEN
T1 - The Post-Stroke Speech Transcription (PSST) Challenge
AU - Gale, Robert C.
AU - Fleegle, Mikala
AU - Fergadiotis, Gerasimos
AU - Bedrick, Steven
N1 - Publisher Copyright:
© European Language Resources Association (ELRA)
PY - 2022
Y1 - 2022
N2 - We present the outcome of the Post-Stroke Speech Transcription (PSST) challenge. For the challenge, we prepared a new data resource of responses to two confrontation naming tests found in AphasiaBank, extracting audio and adding new phonemic transcripts for each response. The challenge consisted of two tasks. Task A asked challengers to build an automatic speech recognizer (ASR) for phonemic transcription of the PSST samples, evaluated in terms of phoneme error rate (PER) as well as a finer-grained metric derived from phonological feature theory, feature error rate (FER). The best model had a 9.9% FER / 20.0% PER, improving on our baseline by a relative 18% and 24%, respectively. Task B approximated a downstream assessment task, asking challengers to identify whether each recording contained a correctly pronounced target word. Challengers were unable to improve on the baseline algorithm; however, using this algorithm with the improved transcripts from Task A resulted in 92.8% accuracy / 0.921 F1, a relative improvement of 2.8% and 3.3%, respectively.
AB - We present the outcome of the Post-Stroke Speech Transcription (PSST) challenge. For the challenge, we prepared a new data resource of responses to two confrontation naming tests found in AphasiaBank, extracting audio and adding new phonemic transcripts for each response. The challenge consisted of two tasks. Task A asked challengers to build an automatic speech recognizer (ASR) for phonemic transcription of the PSST samples, evaluated in terms of phoneme error rate (PER) as well as a finer-grained metric derived from phonological feature theory, feature error rate (FER). The best model had a 9.9% FER / 20.0% PER, improving on our baseline by a relative 18% and 24%, respectively. Task B approximated a downstream assessment task, asking challengers to identify whether each recording contained a correctly pronounced target word. Challengers were unable to improve on the baseline algorithm; however, using this algorithm with the improved transcripts from Task A resulted in 92.8% accuracy / 0.921 F1, a relative improvement of 2.8% and 3.3%, respectively.
KW - anomia
KW - aphasia
KW - automatic speech recognition
KW - speech language pathology assessment
UR - http://www.scopus.com/inward/record.url?scp=85145873865&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85145873865&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:85145873865
T3 - Proceedings - 4th RaPID Workshop: Resources and Processing of Linguistic, Para-Linguistic and Extra-Linguistic Data from People with Various Forms of Cognitive/Psychiatric/Developmental Impairments, as part of the 13th Edition of the Language Resources and Evaluation Conference, LREC 2022
SP - 41
EP - 55
BT - Proceedings - 4th RaPID Workshop
A2 - Kokkinakis, Dimitrios
A2 - Themistocleous, Charalambos K.
A2 - Fors, Kristina Lundholm
A2 - Tsanas, Athanasios
A2 - Fraser, Kathleen C.
PB - European Language Resources Association (ELRA)
T2 - 4th RaPID Workshop: Resources and Processing of Linguistic, Para-Linguistic and Extra-Linguistic Data from People with Various Forms of Cognitive/Psychiatric/Developmental Impairments, RAPID 2022
Y2 - 25 June 2022
ER -