Speaker intonation adaptation for transforming text-to-speech synthesis speaker identity

Mahsa Sadat Elyasi Langarani, Jan Van Santen

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)

Abstract

In this study, we propose a new intonation adaptation method to transform the perceived identity of a Text-To-Speech system to that of a target speaker with a small amount of training data. In the proposed method, during training we fit parametrized accent and phrase curves to parallel recordings of the target speaker F0 curves, and estimate the parameters of a mapping between the corresponding parameter spaces. During test, we fit the accent and phrase curves to the source utterances, apply the mapping, and create an F0 contour from the mapped accent and phrase curves. We compare the proposed method with a baseline adaptation method in which the source F0 contour is transformed linearly such that the per-utterance mean and variance of the target F0 contour is left unaltered. Perceptual tests showed that the proposed method was better than the baseline method in two subjective tests that assess similarity to the target speaker and speech quality, respectively.

Original languageEnglish (US)
Title of host publication2015 IEEE Workshop on Automatic Speech Recognition and Understanding, ASRU 2015 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages116-123
Number of pages8
ISBN (Print)9781479972913
DOIs
StatePublished - Feb 10 2016
EventIEEE Workshop on Automatic Speech Recognition and Understanding, ASRU 2015 - Scottsdale, United States
Duration: Dec 13 2015Dec 17 2015

Other

OtherIEEE Workshop on Automatic Speech Recognition and Understanding, ASRU 2015
CountryUnited States
CityScottsdale
Period12/13/1512/17/15

Fingerprint

Speech synthesis

Keywords

  • Adaptation
  • Intonation modeling
  • Prosody
  • Text-to-Speech synthesis

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Networks and Communications
  • Computer Vision and Pattern Recognition

Cite this

Langarani, M. S. E., & Van Santen, J. (2016). Speaker intonation adaptation for transforming text-to-speech synthesis speaker identity. In 2015 IEEE Workshop on Automatic Speech Recognition and Understanding, ASRU 2015 - Proceedings (pp. 116-123). [7404783] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ASRU.2015.7404783

Speaker intonation adaptation for transforming text-to-speech synthesis speaker identity. / Langarani, Mahsa Sadat Elyasi; Van Santen, Jan.

2015 IEEE Workshop on Automatic Speech Recognition and Understanding, ASRU 2015 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2016. p. 116-123 7404783.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Langarani, MSE & Van Santen, J 2016, Speaker intonation adaptation for transforming text-to-speech synthesis speaker identity. in 2015 IEEE Workshop on Automatic Speech Recognition and Understanding, ASRU 2015 - Proceedings., 7404783, Institute of Electrical and Electronics Engineers Inc., pp. 116-123, IEEE Workshop on Automatic Speech Recognition and Understanding, ASRU 2015, Scottsdale, United States, 12/13/15. https://doi.org/10.1109/ASRU.2015.7404783
Langarani MSE, Van Santen J. Speaker intonation adaptation for transforming text-to-speech synthesis speaker identity. In 2015 IEEE Workshop on Automatic Speech Recognition and Understanding, ASRU 2015 - Proceedings. Institute of Electrical and Electronics Engineers Inc. 2016. p. 116-123. 7404783 https://doi.org/10.1109/ASRU.2015.7404783
Langarani, Mahsa Sadat Elyasi ; Van Santen, Jan. / Speaker intonation adaptation for transforming text-to-speech synthesis speaker identity. 2015 IEEE Workshop on Automatic Speech Recognition and Understanding, ASRU 2015 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2016. pp. 116-123
@inproceedings{2da4e59e0e6744f49ab389788ec70980,
title = "Speaker intonation adaptation for transforming text-to-speech synthesis speaker identity",
abstract = "In this study, we propose a new intonation adaptation method to transform the perceived identity of a Text-To-Speech system to that of a target speaker with a small amount of training data. In the proposed method, during training we fit parametrized accent and phrase curves to parallel recordings of the target speaker F0 curves, and estimate the parameters of a mapping between the corresponding parameter spaces. During test, we fit the accent and phrase curves to the source utterances, apply the mapping, and create an F0 contour from the mapped accent and phrase curves. We compare the proposed method with a baseline adaptation method in which the source F0 contour is transformed linearly such that the per-utterance mean and variance of the target F0 contour is left unaltered. Perceptual tests showed that the proposed method was better than the baseline method in two subjective tests that assess similarity to the target speaker and speech quality, respectively.",
keywords = "Adaptation, Intonation modeling, Prosody, Text-to-Speech synthesis",
author = "Langarani, {Mahsa Sadat Elyasi} and {Van Santen}, Jan",
year = "2016",
month = "2",
day = "10",
doi = "10.1109/ASRU.2015.7404783",
language = "English (US)",
isbn = "9781479972913",
pages = "116--123",
booktitle = "2015 IEEE Workshop on Automatic Speech Recognition and Understanding, ASRU 2015 - Proceedings",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - GEN

T1 - Speaker intonation adaptation for transforming text-to-speech synthesis speaker identity

AU - Langarani, Mahsa Sadat Elyasi

AU - Van Santen, Jan

PY - 2016/2/10

Y1 - 2016/2/10

N2 - In this study, we propose a new intonation adaptation method to transform the perceived identity of a Text-To-Speech system to that of a target speaker with a small amount of training data. In the proposed method, during training we fit parametrized accent and phrase curves to parallel recordings of the target speaker F0 curves, and estimate the parameters of a mapping between the corresponding parameter spaces. During test, we fit the accent and phrase curves to the source utterances, apply the mapping, and create an F0 contour from the mapped accent and phrase curves. We compare the proposed method with a baseline adaptation method in which the source F0 contour is transformed linearly such that the per-utterance mean and variance of the target F0 contour is left unaltered. Perceptual tests showed that the proposed method was better than the baseline method in two subjective tests that assess similarity to the target speaker and speech quality, respectively.

AB - In this study, we propose a new intonation adaptation method to transform the perceived identity of a Text-To-Speech system to that of a target speaker with a small amount of training data. In the proposed method, during training we fit parametrized accent and phrase curves to parallel recordings of the target speaker F0 curves, and estimate the parameters of a mapping between the corresponding parameter spaces. During test, we fit the accent and phrase curves to the source utterances, apply the mapping, and create an F0 contour from the mapped accent and phrase curves. We compare the proposed method with a baseline adaptation method in which the source F0 contour is transformed linearly such that the per-utterance mean and variance of the target F0 contour is left unaltered. Perceptual tests showed that the proposed method was better than the baseline method in two subjective tests that assess similarity to the target speaker and speech quality, respectively.

KW - Adaptation

KW - Intonation modeling

KW - Prosody

KW - Text-to-Speech synthesis

UR - http://www.scopus.com/inward/record.url?scp=84964555662&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84964555662&partnerID=8YFLogxK

U2 - 10.1109/ASRU.2015.7404783

DO - 10.1109/ASRU.2015.7404783

M3 - Conference contribution

AN - SCOPUS:84964555662

SN - 9781479972913

SP - 116

EP - 123

BT - 2015 IEEE Workshop on Automatic Speech Recognition and Understanding, ASRU 2015 - Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

ER -