This article is about the Synthesizer V Studio software known as a voice database. If you are looking for the Synthesizer V concept then click here.
History[]
On November 23, 2022, Dreamtonics posted a poem on bilibili along with an announcement that a new song was expected for November 24 at 16:00 GMT+8. Details were not specified but the poem seemed to describe a band, with the singer seeming to be portrayed as mighty and commanding, hinting at a new voice database. The following day, Dreamtonics announced the development of two new voice databases, a male and female pair known as Xuan Yu and Cong Zheng. Both received their own individual demos. The release date was not yet known but was said to be in the near future.[1] Xuan Yu released simultaneously on Dreamtonics International, Taobao, and WeChat Mini Program on December 12.[2] From then until December 25, Xuan Yu and Cong Zheng would be available at a discounted price via Taobao and WeChat. Individually, their discounted price was set to 479 yuan compared to the regular 599 yuan. For their own bundles with the Synthesizer V Studio Pro editor, the discount price was set to 899 yuan compared to the regular 999 yuan.[3]
Updates[]
A free update for Xuan Yu was released following the Synthesizer V Studio 1.8.1b1 update. This improved the neural vocoder for male vocals in general, which results in an enhanced sound quality in the bass range.
Final version update compatible for Synthesizer V Studio 1.8.1 which improves the neural vocoder for all male vocals, including Xuan Yu, resulting in an enhanced sound quality in the bass range.
A free update for Xuan Yu was released following the Synthesizer V Studio 1.9.1b1 update. This implemented the new rap feature available for the English and Mandarin languages. Cantonese Chinese was added to the Cross-lingual Singing Synthesis feature as well as general improvements to the overall function, resulting in more accurate pronunciation that sounds closer to a native speaker when singing foreign words.
A free update for Xuan Yu was released following the Synthesizer V Studio 1.9.0b2 update. This version of the vocal allowed for compatibility with the engine's second beta, which focused on rectifying several design issues identified by the user base in the 1.9.0b1 post-release survey.
Final version update compatible for Synthesizer V Studio 1.9.0 which enables Xuan Yu to use the rap feature (English and Mandarin Chinese only) and allows for Cantonese Chinese Cross-lingual Singing Synthesis if the beta versions were not installed previously.
A free update for Xuan Yu was released following the Synthesizer V Studio 1.10.0b1 update. This enhanced pitch generation by utilizing Reinforcement Learning from Human Feedback (RLHF), translating to fewer off-key errors, making better use of vibrato, and creating more expressive transitions between notes. This also allowed for the use of the new "Enhancement" slider in the AI Retakes panel, which controlled the level of changes introduced by RLHF. For Synthesizer V Studio Pro users, feedback submission was enabled, allowing users to consensually share their preferred takes with only Dreamtonics for research and development purposes, where models trained with this feedback would be deployed in future voice database updates. Users may also opt out of this feature if undesired.
A free update for Xuan Yu was released following the Synthesizer V Studio 1.10.0b2 update, which was the second beta version following version 1.10.0b1 that introduced enhancements to voice databases and integrated the latest user feedback via Reinforcement Learning from Human Feedback (RLHF). Regarding the editor, numerous reported issues were addressed for a more fluid experience. A new generation of pitch models were trained using the extensive feedback from users over the past month, and the new models offered greater accuracy in voice timbre reproduction, as well as a reduction in mispronunciations. Dreamtonics also focused on issues from the previous beta release that made Vocal Modes seem less distinct on certain vocals, ensuring clearer differentiation in the 1.10.0b2 update. The latest update also introduced a new feature in the Note Properties Panel that allowed for even division of a note into multiple syllables, proving to be a useful tool for songs with faster tempos.
Xuan Yu AI v103 - October 19, 2023[19]
Final version update compatible for Synthesizer V Studio 1.10.0 which enables Xuan Yu to use Reinforcement Learning from Human Feedback (RLHF) as well as other enhancements (as listed in v103b1 and v103b2's notes) if the beta version was not installed previously.
Xuan Yu AI v104 - October 26, 2023
A free update for Xuan Yu was released that added support for the Japanese language to the rap feature.
Xuan Yu AI v105b1 - December 18, 2023[20]
A free update for Xuan Yu was released following the Synthesizer V Studio 1.11.0b2 update. This implemented support for the Spanish language in the Cross-lingual Singing Synthesis feature.
Xuan Yu AI v105 - April 4, 2024[21]
Final version update compatible for Synthesizer V Studio 1.11.0 which enables Xuan Yu to sing in Spanish through Cross-lingual Singing Synthesis if the beta version was not installed previously.
Voice Database Information[]
Demonstrations[]
| Demonstrations | |
|---|---|
|
| |
| ę äŗå¤Ŗē ä½ēå¾éæ (GĆ¹shƬ TĆ i DuĒn YĆŗshÄng HÄn ChĆ”ng) |  bilibili
|
| Dreamtonics Recommended Songs Playlist[22] | |
|---|---|
|
| |
| å¤ (YĆØ) | bilibili
|
| å¦å (RĆŗ DÄo) | bilibili
|
| åč½¬ęŖę„ (FĒn ZhuĒn WĆØilĆ”i) | bilibili
|
| ę øēéå ē² (HĆ© BĆ o HuĆ² YuĆ”n JiĒ) ft. Weina[23] | bilibili
|
| ę¾é¹¤äŗ (FĆ ng HĆØ TĆng) | bilibili
|
| ę¶ä»£ (ShĆdĆ i) | bilibili
|
| å»äŗŗē“ę¢¦ (ShĒrĆ©n ChÄ« MĆØng) ft. Cong Zheng - English Cross-lingual Singing Synthesis | bilibili
|
| ę°éøēå«å§¬ (XÄ«n BĆ wĆ”ng BiĆ© JÄ«) ft. Cheng Xiao | bilibili
|
Voice Databases[]
Product Information
Optimum
Genre: Rock, Pop, Light Rock, Ballads Range: D#3 ~ D4[24]
Versions
Lite Version/FLT Available?: No Starter Pack Available?: Yes Web Version Available?: No
Package details as noted:
Xuan Yu is a masculine Mandarin Chinese voice database. As expected of an AI voice database, this enables him to sing with a more human-like, natural, and realistic voice. He is a synthetic voice created with the image of a rock singer, a first attempt for Dreamtonics, and was designed to have a mature, powerful, and textured voice.
Vocal traits as noted:
- Xuan Yu's voice was described as solid and powerful with skilled voice control.
- He was noted to be suitable for works with strong rhythm styles like rock, and he can deeply express the rich emotion of songs with various strong and husky long notes, natural transitions, and emotive pitch articulations. He was made to capture the strong impression left by Chinese rock singers.
- With Diffusion Probabilistic Models (DPM), Xuan Yu can sing more realistically through changes in volume and breath, further enriching singing details, making each singing unique, and providing creators with more options. Through this mode, he can use AI Retakes to create multiple "takes" of a sung section to find an ideal vocal without the need for "laborious tuning". This feature is only available in the Synthesizer V Studio Pro Editor.
Phonetic notes as noted:
- Phoneme format: X-SAMPA
- Through Cross-lingual Singing Synthesis, Xuan Yu can sing in not only Mandarin Chinese but also in English and Japanese. Since the release of his v102b1 update on April 18, 2023, he can also sing in Cantonese Chinese with this feature; since his v105b1 update on December 18, he can also sing in Spanish with this feature. Cross-lingual Singing Synthesis is only available in the Synthesizer V Studio Pro Editor.
Vocal Modes as noted (Synthesizer V Studio Pro Editor only):
- Xuan Yu comes with five variations:
- "Soft"
- "Light"
- "Passionate"
- "Solid"
- "Open"
License agreements as noted:
- Dreamtonics Co., Ltd.'s End User License Agreement - Japanese
- Dreamtonics Co., Ltd.'s End User License Agreement - English
- Dreamtonics Co., Ltd.'s End User License Agreement - Simplified Chinese
Voice database sample
- Xuan Yu AI SolfĆØge:
File:Xuan Yu AI Solfege.ogg- Vocal Mode SolfĆØges:
- "Soft":
File:Xuan Yu AI Soft.ogg - "Light":
File:Xuan Yu AI Light.ogg - "Passionate":
File:Xuan Yu AI Passionate.ogg - "Solid":
File:Xuan Yu AI Solid.ogg - "Open":
File:Xuan Yu AI Open.ogg
- "Soft":
- Vocal Mode SolfĆØges:
References[]
- ā https://weibo.com/7752303641/MgvVUdzd6
- ā https://twitter.com/dreamtonics_en/status/1602226838583267329
- ā https://t.bilibili.com/738722619768963109
- ā https://dreamtonics.com/en/synthesizer-v-studio-1-8-1b1-update/
- ā https://twitter.com/dreamtonics_en/status/1622505781265563648
- ā https://dreamtonics.com/synthesizer-v-studio-1-8-1-final-update/
- ā https://twitter.com/dreamtonics_en/status/1626100512675500033
- ā https://dreamtonics.com/synthesizer-v-studio-1-9-0b1-update-rap-cantonese-and-more/
- ā https://twitter.com/dreamtonics_en/status/1648235081608343552
- ā https://dreamtonics.com/synthesizer-v-studio-1-9-0b2-update/
- ā https://twitter.com/dreamtonics_en/status/1660888307482656771
- ā https://dreamtonics.com/synthesizer-v-studio-1-9-0-final-update/
- ā https://twitter.com/dreamtonics_en/status/1671427830629154816
- ā https://twitter.com/dreamtonics_en/status/1686648078571655168
- ā https://t.bilibili.com/825177687051993121
- ā https://dreamtonics.com/synthesizer-v-studio-1-10-0b1-update-enhancing-pitch-generation-with-user-feedback/
- ā https://twitter.com/dreamtonics_en/status/1707304150059639108
- ā https://dreamtonics.com/synthesizer-v-studio-1-10-0b2-update/
- ā https://dreamtonics.com/synthesizer-v-studio-1-10-0-final-update/
- ā https://dreamtonics.com/synthesizer-v-studio-1-11-0b2-update/
- ā https://dreamtonics.com/synthesizer-v-studio-1-11-0-update/
- ā https://space.bilibili.com/593592382/favlist?fid=1655788882 - These songs were featured in an official bilibili playlist featuring demos and recommended songs. Certain users were invited to create songs using earlier versions of the voice database and release the songs in sequence after the product's release as a complement to official demo tracks. It should also be noted that certain songs may not be involved with the early invitation and were added based on the company's preference. Only songs that feature Xuan Yu as a main vocalist would be added to this demo table.
- ā Weina uses Cross-lingual Singing Synthesis for Japanese and English. Xuan Yu only sings in Mandarin Chinese.
- ā https://img.alicdn.com/imgextra/i3/54187543/O1CN01e733Zw25akRWer4jY_!!54187543.png - Xuan Yu initially did not have a range listed upon release. This was added to his Taobao product page as of March 15, 2024