Speaker-adaptive neural vocoders for parametric speech synthesis systems

Authors: Eunwoo Song, Jin-Seob Kim, Kyungguen Byun, Hong-Goo Kang
Date: 28 Jun 2019 (last updated 28 Oct 2020)

• Abstract
This paper proposes speaker-adaptive neural vocoders for parametric text-to-speech (TTS) systems. Recently proposed WaveNet-based neural vocoding systems successfully generate a time sequence of speech signal with an autoregressive framework. However, it remains a challenge to synthesize high-quality speech when the amount of a target speaker's training data is insufficient. To generate more natural speech signals with the constraint of limited training data, we propose a speaker adaptation task with an effective variation of neural vocoding models. In the proposed method, a speaker-independent training method is applied to capture universal attributes embedded in multiple speakers, and the trained model is then optimized to represent the specific characteristics of the target speaker. Experimental results verify that the proposed TTS systems with speaker-adaptive neural vocoders outperform those with traditional source-filter model-based vocoders and those with WaveNet vocoders, trained either speaker-dependently or speaker-independently. In particular, our TTS system achieves 3.80 and 3.77 MOS for the Korean male and Korean female speakers, respectively, even though we use only ten minutes' speech corpus for training the model.

• Published in: MMSP 2020
• Preprinted version: 1811.03311

Table of Contents

• Speech analysis and synthesis
• Vocoding model: WaveNet vocoder [1], ExcitNet vocoder [2]
• Text-to-speech
• Vocoding model: WaveNet vocoder, ExcitNet vocoder

Speech analysis and synthesis

Korean female speaker


RAW
WaveNet vocoder
Speaker-dependent	Speaker-independent	Speaker-adaptive
ExcitNet vocoder
Speaker-dependent	Speaker-independent	Speaker-adaptive

Text-to-speech

Korean female speaker


RAW
WaveNet vocoder
Speaker-independent	Speaker-independent	Speaker-adaptive
ExcitNet vocoder
Speaker-dependent	Speaker-independent	Speaker-adaptive

References

[1] A. Tamamori, T. Hayashi, K. Kobayashi, K. Takeda, and T. Toda, “Speaker-dependent WaveNet vocoder,” in Proc. INTERSPEECH, 2017, pp. 1118–1122.
[2] E. Song, K. Byun, and H.-G. Kang, “Excitnet vocoder: A neural excitation model for parametric speech synthesis systems,” in Proc. EUSIPCO, 2019, pp. 1179-1183.

Acknowledgements

Work performed with nVoice team, Clova Voice, Naver Corp.

Citation

@inproceedings{song2020speaker,
    title={Speaker-adaptive neural vocoders for parametric speech synthesis systems},
    author={Song, Eunwoo and Kim, Jin-Seob and Byun, Kyungguen and Kang, Hong-Goo},
    booktitle={Proc. MMSP},
    year={2020}
  }