Neural text-to-speech with a modeling-by-generation excitation vocoder

Authors: Eunwoo Song, Min-Jae Hwang, Ryuichi Yamamoto, Jin-Seob Kim, Ohsung Kwon, Jae-Min Kim
Date: 22 Apr 2020 (last updated 30 Oct 2020)

• Abstract
This paper proposes a modeling-by-generation (MbG) excitation vocoder for a neural text-to-speech (TTS) system. Recently proposed neural excitation vocoders can realize qualified waveform generation by combining a vocal tract filter with a WaveNet-based glottal excitation generator. However, when these vocoders are used in a TTS system, the quality of synthesized speech is often degraded owing to a mismatch between training and synthesis steps. Specifically, the vocoder is separately trained from an acoustic model front-end. Therefore, estimation errors of the acoustic model are inevitably boosted throughout the synthesis process of the vocoder back-end. To address this problem, we propose to incorporate an MbG structure into the vocoder's training process. In the proposed method, the excitation signal is extracted by the acoustic model's generated spectral parameters, and the neural vocoder is then optimized not only to learn the target excitation's distribution but also to compensate for the estimation errors occurring from the acoustic model. Furthermore, as the generated spectral parameters are shared in the training and synthesis steps, their mismatch conditions can be reduced effectively. The experimental results verify that the proposed system provides high-quality synthetic speech by achieving a mean opinion score of 4.57 within the TTS framework.

• Published in: INTERSPEECH 2020
• Preprinted version: 2008.00132

• Acoustic model
  • Tacotron-style model with phoneme alignment [1]
• Vocoding model
  • Proposed MbG-ExcitNet
  • Baseline: WaveNet and ExcitNet trained with ground-truth acoustic parameters [2, 3]
  • Baseline: G-WaveNet and G-ExcitNet trained with generated acoustic parameters [4]

TTS demos

Sample1
Raw	Proposed MbG-ExcitNet
Baseline WaveNet	Baseline ExcitNet
Baseline G-WaveNet	Baseline G-ExcitNet
Sample2
Raw	Proposed MbG-ExcitNet
Baseline WaveNet	Baseline ExcitNet
Baseline G-WaveNet	Baseline G-ExcitNet
Sample3
Raw	Proposed MbG-ExcitNet
Baseline WaveNet	Baseline ExcitNet
Baseline G-WaveNet	Baseline G-ExcitNet
Sample4
Raw	Proposed MbG-ExcitNet
Baseline WaveNet	Baseline ExcitNet
Baseline G-WaveNet	Baseline G-ExcitNet

References

[1] T. Okamoto, T. Toda, Y. Shiga, and H. Kawai, “Tacotron-based acoustic model using phoneme alignment for practical neural text-to-speech systems,” in Proc. ASRU, 2019, pp. 214–221.
[2] A. Tamamori, T. Hayashi, K. Kobayashi, K. Takeda, and T. Toda, “Speaker-dependent WaveNet vocoder,” in Proc. INTERSPEECH, 2017, pp. 1118–1122.
[3] E. Song, K. Byun, and H.-G. Kang, “Excitnet vocoder: A neural excitation model for parametric speech synthesis systems,” in Proc. EUSIPCO, 2019, pp. 1179-1183.
[4] J. Shen, R. Pang, R. J. Weiss, M. Schuster, N. Jaitly, Z. Yang, Z. Chen, Y. Zhang, Y. Wang, R. Skerrv-Ryan et al., “Natural TTS synthesis by conditioning WaveNet on mel spectrogram predictions,” in Proc. ICASSP, 2018, pp. 4779–4783.

Acknowledgements

Work performed with nVoice team, Clova Voice, Naver Corp.

Citation

@inproceedings{song2020neural,
    title={Neural text-to-speech with a modeling-by-generation excitation vocoder},
    author={Song, Eunwoo and Hwang, Min-Jae and Yamamoto, Ryuichi and Kim, Jin-Seob and Kwon, Ohsung and Kim, Jae-Min},
    booktitle={Proc. INTERSPEECH},
    pages={3570--3574},
    year={2020}
  }