center for advanced sound technologies, yamaha corporation vocaloid commercial singing synthesizer...
TRANSCRIPT
Center for Advanced Sound Technologies, Yamaha
Corporation
VOCALOIDCommercial singing synthesizer based on sample
concatenation
Hideki Kenmochi, Hayato OhshitaCenter for Advanced Sound Technologies, Yamaha Corporation, Japan
Center for Advanced Sound Technologies, Yamaha
Corporation
System Diagram
(A) Score Editor
SynthesisScore
ConcatenationSample
Selection
Lyrics Note
(B) SingerLibrary
SynthesisOutput
(C) Synthesis Engine
•Diphones•sustained portions(Multiple pitches)
•Timing adjustment•Pitch transposition•Timbre manipulation
Center for Advanced Sound Technologies, Yamaha
Corporation
Synthesis Score
# s s I I I N N @ @ @ s s Q Q Q N N #Phonetictrack
“Sing a song” [s I N @ s Q N]
Note ON[s I N]
Note ON [@]
Note ON[s Q N]
Pitchtrack
s I N @ s Q Ns I N @ s Q N
Center for Advanced Sound Technologies, Yamaha
Corporation
Concatenation
s I I I N
Timbre(Spectralenvelope)
Useas it is
Last Frame First Frame
Fluctuations from sustained portion added
Interpolation
LocalSpectrum
Center for Advanced Sound Technologies, Yamaha
Corporation
Pitch transposition & timbre manipulation
Waveform FFT
Peak MarkingSample Pitch
(Pre-analyzed)
Pitch conversion rate
ScalingIFFT&
Windowing&Overlapping
AmplitudeModification
Target Pitch(from Score)
SynthesisOutput
STFT
STFT
SpectralEnvelope
Center for Advanced Sound Technologies, Yamaha
Corporation
Pitch transposition & timbre manipulation
Amp
freq
Spectral Envelope
H0 H1 H2 H3 H4 H5 H6
Amp H0 H1 H2 H4 H5 H6
freq
Center for Advanced Sound Technologies, Yamaha
Corporation
Score Editor
Dedicated environment for singing synthesis Note / Lyric input Vibrato setting Control over synthesis parameters
Center for Advanced Sound Technologies, Yamaha
Corporation
Realtime keyboard performance mode
Lyrics input in advance Keyboard triggers note
[Video Clip]
Center for Advanced Sound Technologies, Yamaha
Corporation
VOCALOID Products
Version 1 (released in 2004)
Version 2 (released in 2007)
LEON LOLA MIRIAM Meiko Kaito
Big-AL
Sweet ANN
Prima Miku
Center for Advanced Sound Technologies, Yamaha
Corporation
Acknowledgements
The basic signal processing technique used in Vocaloid is developed though a joint research project between Yamaha Corporation and Music Technology Group (MTG), Universitari Pompeu Fabra, Barcelona.
The authors would like to thank the staff at MTG, especially to Mr. Jordi Bonada and Dr. Alex Loscos, for their contribution to Vocaloid.