This article is about the Synthesizer V Studio software known as a voice database. If you are looking for the Synthesizer V character then click here.
History[]
E-CAPSULE Co. Ltd. had initially approached Yamaha Corporation with the intent of producing XIA YU YAO for the VOCALOID3 engine, however, the two corporations had differing opinions regarding the sales of the potential voicebank, which led to her production as an UTAU instead. This information was not made known until January 17, 2015, when comments posted to VOCALOID3 Xin Hua's Facebook group revealed YU YAO's relation to the VOCALOID franchise. A tuner for Xin Hua's demo songs further clarified the claim.[1][2] YU YAO was released by VOICEMITH, a planning team under E-CAPSULE, with a CVVC+VC Japanese voicebank and a CV-VC Mandarin Chinese voicebank on November 10, 2014, later receiving updated voicebanks and additional "append" Mandarin Chinese voicebanks (OUTSIDE and INSIDE).
2022[]
Q2[]
On May 27, 2022, VOICEMITH posted an announcement written in binary code that translated to "YU YAO AI sound library crowdfund preparation". The image that accompanied the post featured a logo of the third character of her name (YAO) next to a standing microphone.[3][4][5] On May 28, another announcement was posted with an image featuring the number 8 and five colored blocks that hinted at her design's color scheme. This time, the text needed to be decoded with Caesar Cipher while using the number 8 to dictate the number of shifts (rotations) needed to solve it. The message translated to "YAO Synthesizer V AI".[6][7] This second tweet was deleted the next day.
On May 31, XIA YU YAO's AI database crowdfund launched on the flyingV platform that was scheduled to end on July 15 at 23:59.[8] The goal was set to 1,500,000 NTD. Due to the deletion of the tweet containing the confirmation of the engine, the statement was redacted. Instead, when asked about what synthesizer YU YAO AI was being made for, VOICEMITH could not reveal more information aside from fact that it would not be for UTAU.[9] Instead, there were hints in the FAQ referencing the system requirements and the mention of Cross-lingual Singing Synthesis, a feature that was available to Synthesizer V AI though the engine was not mentioned, though the Cross-lingual Singing Synthesis portion was later redacted, only confirming that the voice would be able to sing in Mandarin Chinese, Japanese, and English. Mi Yang would be reprising the role of YU YAO's voice provider while izumi returned as her official illustrator. YU YAO received a brand new design for her AI version and her mock boxart revealed that she would be referred to as both "XIA YU YAO" (located on the spine) and "YAO" (located on the front). Sixteen types of pledges were offered, three of which were early bird tiers that were available until June 7:[10]
- Pure sponsorship with no rewards.
- A physical package of XIA YU YAO AI with an original song download card (early and regular supporters).
- 2 physical packages of XIA YU YAO AI with 2 original song download cards (early and regular supporters).
- A physical package of XIA YU YAO AI with an original song download card, a standee, a crossbody bag, and a mug (early and regular supporters).
- A physical package of XIA YU YAO AI with an original song download card, a standee, a crossbody bag, a mug, a thank you message from Mi Yang, and a commemorative poster with izumi's signature.
- 2 physical packages of XIA YU YAO AI with 2 original song download cards, a standee, a crossbody bag, a mug, a thank you message from Mi Yang, a commemorative poster with izumi's signature, credit in the co-production list, and a nightlight.
- A commemorative power bank.
- Clock and ticket card sets.
- Phone grip and physical "Black & White" single set.
- Blanket and pillow sets.
- Towel, drawstring bag, soft magnet, bookmark, badge, and paper coaster set.
- Dual digital albums and lyric card sets.
- "Sincere" and "Music Collection" digital albums and postcard sets.
- "Happiness Trilogy" physical album and lyrics book set.
The crowdfund rewards were expected to be delivered in November 2022, marking this as her tentative release date.[11] Her voice library was expected to be available for sale in the future, though the price was undetermined at the time.[12] On June 1, a pledge option for digital versions of XIA YU YAO AI's voice library was made available for early bird and regular backers.[13] An FAQ was also posted on VOICEMITH's social media, which included concerns regarding pledges from fans in Mainland China. Chinese fans expressed difficulty with using the flyingV platform and noted that the payment options were not viable for them. VOICEMITH declared that they were working on a plan for Chinese fans. There was also a clarification regarding the engine YU YAO was intended to be made for. VOICEMITH explained that they were not able to confirm the engine until after the crowdfund successfully reached the goal and restated that the library would be made for neither UTAU nor VOCALOID.[14]
On June 8, VOICEMITH released a statement on bilibili noting that because the crowdfunding event was difficult to control, they would be waiting for it to reach its goal before launching the pre-order event in Mainland China.[15] It was also stressed that the digital version would allow users to quickly receive the software through email and save on shipping fees.[16] On June 9, VOICEMITH released another statement in response to concerns from Chinese fans. They noted that they could not modify the display method of the supported delivery areas by themselves after asking the crowdfunding platform directly and apologized for causing any doubts regarding this issue, noting that they would pay more attention in the future. Regarding the pre-order event that would be launched after the crowdfunding period, it would be opened for users in Mainland China, Macau, and Hong Kong via Taobao.[17]
On June 13, VOICEMITH confirmed that a final decision had not been made regarding the discontinuation of YU YAO's UTAU voicebanks, but noted that they were considering stopping it after a period of time.[18] On June 28, it was announced that the main goal was reduced to 1,100,000 NTD instead. This change did not affect existing backers.[19] Another reward was added that included a hardcover official artbook, A2 20-piece poster bundle, and a throw pillow.[20]
Q3[]
After reaching 73.42% of the crowdfunding goal on June 29, the official XIA YU YAO Twitter made a follow-up announcement on July 4 stating that the voice synthesizer engine that was chosen was decided, however, VOICEMITH could not announce it during the crowdfunding period due to contractual constraints.[21][22] They suggested using hints from their shared tweets in the last few days such as the "Great!" response to Dreamtonics Co., Ltd.'s HDVM technology announcement.[23] On July 5, support via PayPal was opened.[24] On July 7, after receiving many comments and support from overseas fans wanting to see the completion of YU YAO's voice database, VOICEMITH was inspired and after internal discussions, it was decided that the goal was lowered again to 1,000,000 NTD.[25] On July 8, the crowdfund reached its goal.[26] The crowdfund ended with a total of 1,299,165 NTD and 331 backers on July 15.
On August 24, VOICEMITH officially announced XIA YU YAO to be in production for Synthesizer V AI and launched pre-orders for her digital copies on their official website. The software was expected to be uploaded to the Taiwan Cloud Space in November.[27] On September 6, YU YAO's official Twitter posted that the recordings for the voice database took place in the YuR Room of Mega Force Recording Studio in Xizhi, New Taipei City. It was noted that this particular room had a 664 square-meter sound-receiving space with a designed reverberation value of about 0.35 seconds, and a 17 square-meter soundproof booth. The first recording work was already completed and the relevant materials were delivered to Dreamtonics for data processing. Two more recordings were expected to be done in September.[28]
On September 19, a second production document released stating that in the previous week, VOICEMITH visted the studio again for the second samplings and that the recordings went well. In addition to continuously increasing the sample data, for this time and the next time, there would be sample recordings of "special voices". VOICEMITH hoped to provide different voice lines in the final package for users to use. A Lite version for YU YAO was planned to be released in the future. The next visit to the recording studio was expected to be the last.[29]
Q4[]
On November 1, it was announced that Mi Yang and the recording engineers were diagnosed with COVID-19 a few days before, delaying the progress of recording. The official XIA YU YAO Twitter account assured everyone that they recovered. In addition, all of the sound samples were prepared a week prior and were submitted to Dreamtonics for processing. Designs drafts of merchandise were provided to manufacturers and production already began. The physical box was also updated in the announcement's image.[30] On November 8, the official XIA YU YAO Twitter account posted that applications for beta testing were open until November 25. Approved testers would be provided a beta voice database to use and were to provide more than two minutes of raw samples and feedback about any issues with the voice database. The test period would run from December 1 to December 10.[31]
On November 24, Synthesizer V Studio was updated to version 1.8.0 and introduced Diffussion Probabilistics Models (DPM).[32] As VOICEMITH wanted YU YAO to be equipped with the newest update to present better singing quality, the beta test version was delayed to mid-December, with the final version to be expected in early January 2023.[33] On November 28, VOICEMITH tweeted that over 300 applicants applied for beta testing the voice database and noted that time was needed to choose the testers.[34] On November 29, VOICEMITH announced that crowdfunding rewards, including the physical box copies of XIA YU YAO, would begin distribution and reminded backers who pledged for the digital and/or physical copy of the voice database that they would not receive it until the production was finished.[35]
On December 21, a newsletter reported that after reviewing the feedback from beta testers and letting Dreamtonics know about the testers' opinions, VOICEMITH would be working on the corrections soon. It was also teased that there were five Vocal Modes: Dark, Soft, Solid, Whisper, and Sweet. It was reconfirmed that Mi Yang's native language was Mandarin Chinese and that she was not good at singing in English and Japanese, thus affecting the voice database's Cross-lingual Singing Synthesis capabilities. VOICEMITH warned that the voice database may not meet the standards of English and Japanese articulation and pronunciation.[36]
2023[]
On January 11, 2023, the official XIA YU YAO Twitter account announced that supporters who pledged for solutions A, C, D, E, and F, they would be able to download and use the voice database early after its completion. Two new original songs, mobile phone wallpapers, and desktop wallpapers would also be released. For those who purchased the physical box for the voice library, these goods would be distributed via the QR code on the music download card. For those who supported the digital download of the voice database, these goods would be sent via email.[37] On January 13, beta testers began to upload works using the voice library.[38] A pre-order page for the digital copies of the voice library was launched and a discounted period was expected to begin on January 31 and end on February 12, pricing it at 1250 NT instead of the regular 2500 NT. However, this page was taken down some time later.
On January 17, it was announced that YAO AI would be released with two different physical boxes: the crowdfund version uses her key art for the outer sleeve and box while the regular edition uses a different illustration on the outer sleeve and the actual box used the key artwork. The crowdfund version would also be packaged with a mobile phone holder and sponsors would receive ten original songs (two brand new and eight self-covers retuned with the AI voice database), four PC wallpapers, and four mobile wallpapers. Sponsors would also be receiving the voice database early on January 18. For the general public, pre-orders would be launching on January 19 and end on February 5, with a discounted price of 1800 NT for the digital version while the physical boxes would not be receiving a discount. The final release was set for January 30.[39] On January 18, a separate store page was launched for overseas users. It was explained that due to the cash flow system of Taiwan stores, overseas credit cards could not be supported, thus the need to use methods provided by PayPal via account transfer and credit card. Based on past experience, there were issues that occurred frequently for overseas shipments, such as delayed receipts, expensive shipping fees, and damaged packaging, so the physical box would only be available for purchase in Taiwan. It was stated that the physical box did not contain a CD and that the voice database would still needed to be obtained via digital download.[40] On January 20, VOICEMITH announced that they were partnering with VC Palace Group to distribute the digital and physical copies of the voice database via Taobao for Mainland Chinese users for a limited time.[41]
On January 27, VOICEMITH started to deliver the activation codes and download links for YAO AI to regular-public customers who preordered her via email.[42] She released on January 30 as intended. In response to a fan on February 23, VOICEMITH stated that the Lite version of XIA YU YAO would not be released. No additional information was provided.[43]
Updates[]
Voice Database Information[]
Demonstrations[]
| Demonstrations[55] | |
|---|---|
|
| |
| ęŗčŖå¤ēč²é³ (YuĆ”nzƬ XiĆ de ShÄngyÄ«n) - Short Versions; Acapella; All Vocal Modes |  YouTube  bilibili
|
| čæ½č± (ZhuÄ« HuÄ), éå§å (ZuƬ XuÄnxiÄo), é¶åŗ¦äøēēøé (LĆngdĆ¹ XiĆ de XiÄngyĆ¹), ę“天 (QĆngtiÄn), ęäøčŖē± (Ći BĆ¹ ZƬyĆ³u) - Short Versions; All Vocal Modes | YouTube bilibili
|
| ēŗÆē½ä¹čŖ (ChĆŗn BĆ”i zhÄ« ShƬ) | YouTube bilibili
|
| éØę (YĒlĆn) | YouTube bilibili
|
| ęč·Æ (XÄ«ng LĆ¹) | YouTube bilibili
|
| Butterfly - English Cross-lingual Singing Synthesis | YouTube bilibili
|
| ēŖčēč (FÄnqiĆ© ChĒo DĆ n) | YouTube bilibili
|
| fictionęØ”ęØ£ (fiction MĆ³yĆ ng) | YouTube bilibili
|
| Genie - English Cross-lingual Singing Synthesis | YouTube bilibili
|
| ęŗčŖå¤ēč²é³ (YuĆ”nzƬ XiĆ de ShÄngyÄ«n) - Full Version | YouTube bilibili
|
Voice Databases[]
- According to the newsletter released on December 21, 2022, since the AI voice database was able to capture and restore the voice provider's vocal range and singing voice, the sampling in the high-pitch range for YAO AI was based on Mi Yang's voice characteristics. It was noted that she was good at singing as a soft voice around the middle range, which would be present in the YAO AI voice library. The original UTAU version, however, was noted to be a very strong voice in the upper range in comparison. If users wanted the Synthesizer V AI version to have a stronger sound in the upper range, it was recommended to adjust the Vocal Modes and tension parameters to try to get a better result.[36]
- According to Dreamtonics, YAO AI had a soft and bright voice full of natural emotion. The voice was rich in vocal variations that would give listeners the feeling of nostalgia when she performed tender songs.[56]
- YAO AI's falsetto range was noted to be above C5.[36]
- According to various beta testers:[57][58]
- CircusP noted that she was versatile.
- Guimian-P said that she was easy and comfortable to use. Her performance was relatively stable, giving people a very fresh and gentle feeling.
- Pigeon noted that she sounded very much like herself with her characteristics intact.
- Kinoko-P said that she was suitable for singing gentle ballads. She had a soft voice described to be tender and long-lasting.
- Hugwalk said that her weaker voice can perform better with lyrical songs with a natural breath and warm, pure voice. However, this can limit her singing style. In suitable genres, the performance was top notch and her overall performance could even keep pace with Mai. He hoped that in future updates, she can regain her advantages in cheerful, exciting, and powerful songs.
- EmpathP noted that her English (via Cross-lingual Singing Synthesis) was considered phenomenal.
- Liuxu said that compared to the UTAU voicebank, the degree of freedom in adjustment is much higher. He was moved to hear her presented in a more modern and effective way.
- ryusouta said that she was a very high-quality vocalist with a precious, soft voice and cute tone. She can suit different genres, noting her to be diverse.
- Kim Sang said that her voice was of excellent quality and can reproduce her own voice characteristics very well. She was easy to tune and effects can be done with simple operations.
- With Diffusion Probabilistic Models (DPM), YAO AI can sing more realistically through changes in volume and breath, further enriching singing details, making each singing unique, and providing creators with more options. Through this mode, she can use AI Retakes to create multiple "takes" of a sung section to find an ideal vocal without the need for "laborious tuning". This feature is only available in the Synthesizer V Studio Pro Editor.
- Phoneme format: X-SAMPA
- Through Cross-lingual Singing Synthesis, YAO AI can sing in not only Mandarin Chinese but also in English and Japanese. Since the release of her v101b1 update on June 21, 2023, she can also sing in Cantonese Chinese with this feature; since her v104b1 update on December 18, she can also sing in Spanish with this feature. Cross-lingual Singing Synthesis is only available in the Synthesizer V Studio Pro Editor.
- YAO AI comes with five variations:[36]
- "Dark": Cold and depressing vocals.
- "Soft": An elegant and gentle singing style.
- "Solid": A loud and full singing voice.
- "Whisper": Has soft and breathy vocals.
- "Sweet": A cute and sweet voice.
- There was a mode for "Vocalfry", but it was only available to beta testers. This mode was not available in the final release.[59]
- OS: Microsoft Windows 11 / 10 / 8.1 (64 bit); Mac OS X 10.11 or higher, Ubuntu 20.04 or higher (64 bit)
- CPU: Windows/Intel Mac/Linux/Intel Core i5
- Memory: 2GB or more
- HDD: 1 GB or more
- Display resolution: 1280 x 800 or higher
- VOICEMITH's End User License Agreement - English & Traditional Chinese; as of January 2023
- VOICEMITH's Voice Library Usage Guidelines - Traditional Chinese; as of January 2023
- VOICEMITH's Character Usage Guidelines - Traditional Chinese; as of November 2018
- YAO AI SolfĆØge:
File:YAO AI Solfege.ogg- Vocal Mode SolfĆØges:
- "Dark":
File:YAO AI Dark.ogg - "Soft":
File:YAO AI Soft.ogg - "Solid":
File:YAO AI Solid.ogg - "Whisper":
File:YAO AI Whisper.ogg - "Sweet":
File:YAO AI Sweet.ogg
- "Dark":
- Vocal Mode SolfĆØges:
References[]
- ā https://www.facebook.com/groups/1531345563810791/posts/1531399817138699/?comment_id=1532066050405409
- ā https://docs.google.com/document/d/1fGYTZs136JzjKIPbgmvF3d_xKTg9ZXuxGjuTj1YjF-Y/edit
- ā https://twitter.com/voicemith/status/1530083242375671809
- ā https://www.facebook.com/voicemith/posts/574223090727643
- ā https://www.youtube.com/post/Ugkxx07enxAhOzbMqOvF8uvPQ7lkrPjkfhUn
- ā
https://twitter.com/voicemith/status/1530436710240485376(archived) - ā https://www.facebook.com/voicemith/posts/574934777323141
- ā https://twitter.com/voicemith/status/1531497943554961408
- ā https://twitter.com/Nimatcha/status/1531516624347361281
- ā https://www.facebook.com/voicemith/posts/576966650453287
- ā https://www.flyingv.cc/projects/31995
- ā https://www.flyingv.cc/projects/31995/faqs
- ā https://twitter.com/voicemith/status/1531857142160195584
- ā https://twitter.com/voicemith/status/1531996690148134912
- ā https://www.bilibili.com/read/cv17002872
- ā https://twitter.com/voicemith/status/1534472759430246400
- ā https://www.bilibili.com/read/cv17018695
- ā https://twitter.com/voicemith/status/1536348088310599680
- ā https://twitter.com/YuyaoOfficial/status/1541650049356025856
- ā https://twitter.com/YuyaoOfficial/status/1541726510125461504
- ā https://twitter.com/YuyaoOfficial/status/1542174353177350144
- ā https://twitter.com/YuyaoOfficial/status/1543917382023778305
- ā https://twitter.com/YuyaoOfficial/status/1542892879714349056
- ā https://twitter.com/YuyaoOfficial/status/1544334058272215041
- ā https://twitter.com/YuyaoOfficial/status/1544977110665551873
- ā https://twitter.com/YuyaoOfficial/status/1545227782438809601
- ā https://twitter.com/YuyaoOfficial/status/1562347917029310466
- ā https://twitter.com/YuyaoOfficial/status/1566989686023028737
- ā https://twitter.com/YuyaoOfficial/status/1571798117137395712
- ā https://twitter.com/YuyaoOfficial/status/1587382706362458112/
- ā https://twitter.com/YuyaoOfficial/status/1589919150629322760
- ā https://dreamtonics.com/en/synthesizer-v-studio-1-8-0-final-update/
- ā https://twitter.com/YuyaoOfficial/status/1596015814682890240
- ā https://twitter.com/YuyaoOfficial/status/1597157929270276097
- ā https://www.flyingv.cc/projects/31995/posts/17146
- ā 36.0 36.1 36.2 36.3 https://twitter.com/YuyaoOfficial/status/1605500445736153089
- ā https://twitter.com/YuyaoOfficial/status/1613117726297591808
- ā https://twitter.com/YuyaoOfficial/status/1613834324247474176 - First beta test upload
- ā https://twitter.com/YuyaoOfficial/status/1615289294725451780
- ā https://twitter.com/YuyaoOfficial/status/1615903102913376261
- ā https://t.bilibili.com/753214217403760723
- ā https://cdn.discordapp.com/attachments/695380056155095051/1068550916443213934/image.png - Copy of example email message
- ā https://twitter.com/shiweimigi/status/1760814253227851835
- ā https://dreamtonics.com/synthesizer-v-studio-1-9-0-final-update/
- ā https://twitter.com/dreamtonics_en/status/1671427830629154816
- ā https://www.bilibili.com/read/cv24826703/
- ā https://twitter.com/dreamtonics_en/status/1686648078571655168
- ā https://t.bilibili.com/825177687051993121
- ā https://dreamtonics.com/synthesizer-v-studio-1-10-0b1-update-enhancing-pitch-generation-with-user-feedback/
- ā https://twitter.com/dreamtonics_en/status/1707304150059639108
- ā https://dreamtonics.com/synthesizer-v-studio-1-10-0b2-update/
- ā https://dreamtonics.com/synthesizer-v-studio-1-10-0-final-update/
- ā https://dreamtonics.com/synthesizer-v-studio-1-11-0b2-update/
- ā https://dreamtonics.com/synthesizer-v-studio-1-11-0-update/
- ā The YAO AI voice database was made available for select beta testers, who released content beginning on January 13, 2023. This demo table compiles a list of songs that VOICEMITH directly shared on their accounts. For a volunteer-contributed compilation playlist of beta tests, see here (YouTube).
- ā https://dreamtonics.com/en/synthesizerv/
- ā https://www.voicemith.com/yaoaivoicebank/
- ā https://www.voicemith.com/user-evaluation/
- ā https://www.youtube.com/watch?v=Kmdo488z514