Synthesizer V AI is the next generation of the Synthesizer V vocal synthesis engine, and update to the Synthesizer V Studio software developed by Dreamtonics. The software was unveiled October 30, 2020 in a press release alongside voice database Saki AI and the announcement of Koharu Rikka.[1][2]
According to the press release, Synthesizer V AI has been upgraded with deep learning artificial intelligence and will allow for switching between traditional and AI singing voices. In addition, it was revealed that Synthesizer V database Saki would be upgraded to the AI version (Saki AI) and would be released freely for a limited time to existing Synthesizer V Saki users. Saki AI will also be receiving a Lite version and will also be available on Synthesizer V Studio Basic for free for users to try.[1]. It was released as free update to Synthesizer V Studio in version 1.1.0 for both Basic and Pro versions on December 25, 2020.[3]
It was noted by VOLOR and Eclipsed Sounds, LLC that Synthesizer V Standard voice databases would need to be recorded at the Dreamtonics studio in Tokyo, Japan.[4] In comparison, AI voice databases use song-based recording and would give a chance to be done remotely, with the machine learning done alone by Dreamtonics.[5] It was also noted that AI voice databases could also sound very different to the Standard counterparts, but can also possibly bring more expression.[6][7]
History[]
2020[]
On October 28, 2020, Dreamtonics revealed the announcement for AI updates to Synthesizer V Studio on BiliBili, however, the announcement was removed shortly after.[8]
Synthesizer V AI was formally announced on October 30, 2020 by Dreamtonics in a press release, as well as the voice databases Saki AI and Koharu Rikka, which were to be distributed by AH-Software Co. Ltd.. [1][2] It was also announced that a crowdfunding campaign for the VOICEROID Tsuina-chan to receive both a standard Synthesizer V Studio and Synthesizer V AI library would be launched.[9][2]
Demonstrations for Synthesizer V AI were uploaded to Dreamtonics and AH-Software Co. Ltd.'s YouTube channels respectively showcasing Saki AI. [10][11]. A comparison video between the standard Synthesizer V Studio Saki vocal and the AI version was also uploaded.[12]
2021[]
Synthesizer V Studio was upgraded to version 1.2.0 on February 18, 2021 with AI voices now regarded as "Gen 2 AI voices". Saki AI was also upgraded to version 104 along with the 1.2.0 update.[13][14]
Synthesizer V Studio was upgraded to version 1.3.0 on June 18 with Synthesizer V AI being upgraded and now referred to as "Gen 3". Updates for Saki AI version 110 and Koharu Rikka version 105 were also released to coincide with the update.[15][16]
On December 23, version 1.5.0 was launched, which brought new comprehensive upgrades to AI-based voice databases. Its major update was "Cross-lingual Singing Synthesis", which allowed for AI voice databases to sing in fluent English, Japanese, and Mandarin regardless of whether the vocalist learned these languages or not. Previously, the libraries were limited to only the language proficiency of the voice providers they were based on. By this time, all released AI vocals would be receiving an update to support cross-lingual synthesis and the new feature was only available for the Pro editor.[17] On the same day, it was announced that all of the Beijing Photek S&T Development Co., Ltd. voice databases would be supported for Synthesizer V AI, but were instead trained by the Standard versions and were not based on new recordings.[18] These specific voice databases were officially known as "Synthesizer V Plus".[19]
2022[]
On February 3, 2022, Kanru Hua noted that the WaveRNN vocoder was swapped with a non-autoregressive architecture in November 2021 and within the following updates, WaveRNN was completely removed. By this point, Synthesizer V AI no longer retained a single component from the first version reveal in 2020.[20] On February 4, according to AH-Software, the Synthesizer V series had been selling much more than expected within a year ever since it became compatible with AI.[21]
On June 30, 2022, "High Dynamics Voice Model" (HDVM) was introduced through a demonstration video, which was first made available in Chinese via bilibili, then English and Japanese via YouTube.[22][23] It was confirmed to be a new feature coming to a future update and was described to be technology that learned the variations of dynamics from natural singing and applying the patterns to synthetic voices. It would replicate the continuous rises and falls of loudness while singing, meaning that it would add nuanced changes in the timbre to vocals rendered through Synthesizer V. This was to demonstrate more accurate and natural voice quality, enabling more realistic expression.
2023[]
On February 28, 2023, Dreamtonics announced that Synthesizer V Studio would soon add Cantonese Chinese as its fourth supported language. This would allow the engine to support both voice libraries dedicated to the language as well as Cross-lingual Singing Synthesis. The company also announced the future support of rap vocals, showing a demo of a new male vocalist rapping in Mandarin Chinese and English. Support for Japanese rap was expected in the future.[24][25] On March 2, Dreamtonics posted a response to fans' concerns with the implementation of Cantonese Chinese and noted that they were checking and fixing the issues with the demonstration clips as reported by the user base. They also noted that Synthesizer V Studio supported the input of lyrics in Cantonese Jyutping, which was the 1993 version of the Cantonese spelling scheme. It was not equivalent to the X-SAMPA phonetic scheme above the lyric notes on the editor. The X-SAMPA phonetic scheme for a Chinese character was also not equivalent to the Pinyin reading of the character.[26] On March 15, after receiving feedback in improving the song to be more in line with Cantonese songwriting habits, Dreamtonics replaced the bilibili version of the debut video, which implemented corrections made to the male vocal's and Feng Yi's demos.[27][28] The rap feature for English and Mandarin Chinese, and the implementation of Cantonese Chinese Cross-lingual Singing Synthesis was officially planned to be fully implemented in Version 1.9.0, with a beta version released on April 18. Dreamtonics mentioned that after receiving valuable feedback, they focused on refining pronunciation for an even better user experience. As for how it worked, they said that when the language is set to Cantonese, all Chinese lyrics will be sung with Cantonese pronunciation. If misread lyrics occurred, users can correct them by typing the romanized form in Jyutping directly. Although the phoneme set is largely based on Mandarin Chinese, several phonemes unique to Cantonese were incorporated.[29]
On July 18, Dreamtonics announced that they began to adapt Reinforcement Learning with Human Feedback (RLHF) to Synthesizer V AI's pitch model. AI Retakes were introduced since version 1.7.0, which randomized generation and allowed users to choose their preferred rendition from a list of options. Building upon this, a pair of singing samples would be generated and presented to the listener, who then compared the two and rated them. The ratings would inform the model's learning until the feedback loop allowed for it to refine the voice generation based on the listener's preference. RLHF-enhanced models were reported to be an improvement over the diffusion model baseline across languages according to internal feedback. RLHF-enhanced models were expected to produce fewer off-tune notes, with singing expressions that were more relevant to the context and better use of vibrato. RLHF-enhanced pitch models were expected to be part of the next update. In addition, there were plans to extend RLHF to rap vocals and voice timbre.[30][31]
On November 5, Eclipsed Sounds, LLC announced that, with guidance from Dreamtonics for its research and development, Synthesizer V Studio will add support for Spanish as a Cross-lingual Singing Synthesis language in a future update, with SAROS, SOLARIA, and ASTERIAN becoming the first vocals to receive it before spreading to other voice databases.[32] Dreamtonics later confirmed this development on a later tweet.[33]
Updates[]
For a full list of updates, see the Synthesizer V Studio page.
Releases[]
Full Versions[]
- Full versions are the paid versions of the voice databases (with the exception of Mai, who is free to download for all Synthesizer V Studio Pro users).*
- This list only includes the Synthesizer V AI voice databases. For the Standard voice databases, see Synthesizer V Studio.
Lite Versions[]
- Lite versions are free versions of the voice databases and are available on Synthesizer V Studio Basic and Pro.
- These are generally monopitch (1 pitch) voice databases.
- General guidelines include mentioning "Lite" as part of an uploads' credit in the title and description, as well as disallowing commercial use. Further guidelines may depend on each voice database and it is encouraged that they are read carefully.
- AI Lite versions are not able to use Synthesizer V Studio Pro's Cross-lingual Singing Synthesis feature.
- This list only includes the lite versions of the Synthesizer V AI voice databases. For the Standard voice databases, see Synthesizer V Studio.
Note: Not every voice database has a Lite version.
Feature-Limited Trials (FLTs)[]
- Feature-Limited Trials (FLTs) versions are free-to-use alternatives compatible with both Synthesizer V Studio Basic and Pro with possible limitations and adjustments made to the voice
- These voice databases may be considered to be test versions prior to the release of the final commercial product.
- These versions may be timed or are restricted to specific engine versions to allow users to experience the latest features and updates.
- General guidelines include mentioning "Synthesizer V AI *PRODUCT NAME* Feature Limited Trial" as part of an uploads' credit in the title and/or description, as well as disallowing commercial use. Further guidelines may depend on each voice database and it is encouraged that they are read carefully.
- FLTs may not be able to use certain Synthesizer V Studio Pro features such as Cross-lingual Singing Synthesis and Vocal Mode.
Note: Not every voice database has an FLT.
Starter Packs[]
- Full versions that can be acquired together with the Synthesizer V Studio Pro at a discount.
- ANiCUTE Bundle Packs, Dreamtonics Bundles, and Taobao Sets consist of the voice databases themselves bundled with the SVS Pro editor in the same digital download; they do not exist as physical bundles.
- Regarding the Dreamtonics dual packs and VOICEMITH bundles, these do not come with the SVS Pro editor. Instead, they feature only the two voice databases available in the packs.
- AH-Software starters (marked with * below) are just standalone voice databases that can be freely downloaded from the AHS Store after redeeming the coupon enclosed within the (physical-only) Synthesizer V Studio Pro Starter Pack. Only one of them can be downloaded per Starter Pack coupon.
- This only lists starter packs for Synthesizer V AI voice databases. For the Standard voice databases, see Synthesizer V Studio.
Note: Not every voice database has a Starter Pack.
![Pro + 1 Vocalist Bundle (Dreamtonics Intl.)[34]](/wiki/File:Bundle-box-art.png "Bundle-box-art.png (288 KB)")
Announced Vocals[]
Discontinued Bundle Packs[]
Unreleased Vocals[]
- The "Synthesizer V Plus" vocals were AI voice databases trained with the Standard versions. These are not considered to be part of the "Synthesizer V AI" brand due to the difference in production method and quality, but have similar functions to a full AI voice database, such as Cross-lingual Singing Synthesis.
Unknown Status[]
Known Problems[]
- Synthesizer V Studio 1.1.0
- Dreamtonics noted that some users confirmed crash reports related to AI voices. This bug affects 2nd, 3rd gen Intel i3/i5/i7 processors and AMD processors from Jaguar to Steamroller series. Dreamtonics has recommended upgrading to Synthesizer V Studio 1.1.1[35]
- Synthesizer V Studio 1.2.0
- Dreamtonics noted that some users are experiencing crashes while updating Saki AI to version 104, they have recommended directly downloading & installing the voice from their website. They are currently looking into this issue and will release a fix as soon as possible.[36]
- Synthesizer V Studio Pro 1.7.0
- Some users complained about unstable voice-quality results and changes when re-rendering in 1.7.0 project files saved under earlier versions of Synthesizer V Studio Pro. On July 22, Dreamtonics explained that by default, the AI Retakes feature favors "rich timbre and expressions over smooth and stable vocalization", resulting in inconsistent behavior relative to previous versions;[37] as a workaround to stabilize the voice output, both Dreamtonics and AH-Software offered these steps below:[38][39]
- Empty the selection.
- Open AI Retakes panel, go to Timbre tab.
- Under Takes -> Global Settings, reduce Expressiveness.
- In addition, it was reported that project files saved in 1.7.0 crashed when opened in 1.6.1.[40]
- Some users complained about unstable voice-quality results and changes when re-rendering in 1.7.0 project files saved under earlier versions of Synthesizer V Studio Pro. On July 22, Dreamtonics explained that by default, the AI Retakes feature favors "rich timbre and expressions over smooth and stable vocalization", resulting in inconsistent behavior relative to previous versions;[37] as a workaround to stabilize the voice output, both Dreamtonics and AH-Software offered these steps below:[38][39]
Additional notes[]
Examples of usage[]
Demonstrations[]
Demo's of Saki AI's voice library were uploaded to Dreamtonics and AH-Software Co. Ltd. YouTube Channels
- Saki AI オリジナルデモ曲「カトレアの咲く頃に」 YouTube Broadcast
- Synthesizer V voice - Regular vs AI YouTube Broadcast
- 【SynthV AI】Synthesizer V Sound Demo【開発中】 YouTube Broadcast
Gallery[]
Media Gallery[]
References[]
- ↑ 1.0 1.1 1.2 Dreamtonics announcement
- ↑ 2.0 2.1 2.2 AH-Software announcement
- ↑ Synthesizer V AI immediate release announcement
- ↑ https://www.youtube.com/watch?v=ET6Kdw15L_8
- ↑ https://twitter.com/OfficialVolor/status/1392293299105370113
- ↑ https://twitter.com/OfficialVolor/status/1392293300447498240
- ↑ https://twitter.com/OfficialVolor/status/1403668594811019267
- ↑
https://t.bilibili.com/449209478440913295 - ↑ https://twitter.com/Tuina_chan_PJ/status/1322058582993465344
- ↑ https://www.youtube.com/watch?v=vEW-Ym3HGdE
- ↑ https://www.youtube.com/watch?v=YaiS9qzTiuU
- ↑ https://www.youtube.com/watch?v=T63KKcG1jBM
- ↑ https://twitter.com/dreamtonics_en/status/1362598004100800519
- ↑ https://www.youtube.com/watch?v=D_cXLpO0qIw
- ↑ https://dreamtonics.com/en/synthesizer-v-studio-1-3-0-update/
- ↑ https://twitter.com/dreamtonics_en/status/1362598004100800519
- ↑ https://dreamtonics.com/en/cross-lingual-support-for-synthesizer-v-ai/
- ↑ https://t.bilibili.com/607328123917360256
- ↑ https://t.bilibili.com/609647028303295976
- ↑ https://twitter.com/khuasw/status/1489132723553906689
- ↑ https://www.ah-soft.com/press/synth-v/20220204.html
- ↑ https://www.bilibili.com/video/BV1Cr4y1M7G1
- ↑ https://www.youtube.com/watch?v=zYey-utSIyw
- ↑ https://www.bilibili.com/video/BV1zs4y1f7QJ/
- ↑ https://www.youtube.com/watch?v=mcJU0Wq-u7w
- ↑ https://t.bilibili.com/768416404500119586
- ↑ http://www.bilibili.com/video/BV1zs4y1f7QJ - "Dreamtonics 已于 3 月 15 日将 Synthesizer V AI 粤语歌声合成技术预览的测试曲目更换为更加符合粤语歌曲创作习惯的版本，感谢各位创作者的关心与鞭策。未来 Dreamtonics 将陆续发布更多关于粤语歌声合成与跨语言合成的信息，敬请期待。"
- ↑ https://t.bilibili.com/773207552189005831
- ↑ https://dreamtonics.com/synthesizer-v-studio-1-9-0b1-update-rap-cantonese-and-more/
- ↑ https://twitter.com/dreamtonics_en/status/1681197141396701184
- ↑ https://youtube.com/watch?v=ZKwGR08kCSk
- ↑ https://www.eclipsedsounds.com/post/spanish-is-coming-synth-v
- ↑ https://twitter.com/dreamtonics_en/status/1721369516020846592
- ↑ Choices include: An Xiao, ASTERIAN, Ayame, Cheng Xiao, Cong Zheng, D-Lin, Eri, Feng Yi, Hayden, Jin, Kevin, Lin Lai, Mo Chen, Natalie, Ninezero, Qing Su, Ritchy, Ryo, Saki AI, Sheena, SOLARIA, Weina, Wei Shu, Xuan Yu, Yuma, or Yun Quan
- ↑ https://twitter.com/dreamtonics_en/status/1342732089712492545
- ↑ https://twitter.com/dreamtonics_en/status/1362607750782414848
- ↑ https://twitter.com/dreamtonics_en/status/1550520904291012609
- ↑ https://twitter.com/dreamtonics_en/status/1550520905868009472
- ↑ https://twitter.com/ahsoft/status/1550385484027035649
- ↑ https://twitter.com/ahsoft/status/1550385882141966336