Daily Archives: June 13, 2024

MBWs World Leaders is a regular series in which we turn the spotlight toward some of the most influential industry figures overseeing key international markets. In this feature, we speak to Kyogu Lee, President of HYBE-owned voice AI company Supertone. World Leaders is supported byPPL.

AI technology is a big priority for HYBE, the South Korean entertainment giant behind K-pop superstars BTS.

Evidence of that arrived in March when HYBEs CEO Jiwon Park consented to have his own voice cloned to demonstrate the capabilities of the companys proprietary AI on its Q1 investor call.

HYBEs so-called voice synthesis technology was developed by Supertone, the AI voice replication software startup in which HYBE acquired a majority stake in a $32 million deal in 2023.

Founded in Seoul in 2020, Supertone claims to be able to create a hyper-realistic and expressive voice that [is not] distinguishable from real humans.

Supertones purported ability to do just that makes the apparent strategy behind HYBEs multi-million dollar investment in the technology a lot clearer, when viewed through the lens of comments shared by HYBE Chairman Bang Si-Hyuk in an interview with Billboardlast year.

I have long doubted that the entities that create and produce music will remain human, saidBang Si-Hyuk.

I dont know how long human artists can be the only ones to satisfy human needs and human tastes. And thats becoming a key factor for my operation and a strategy for HYBE.

By acquiring Supertone, HYBE has also brought into the fold the startups co-founder and President Kyogu Lee, a widely respected AI expert with a PhD in Computer-Based Music Theory and Acoustics from Stanford University.

In addition to heading up Supertone at HYBE, he leads applied research at the Artificial Intelligence Institute at Seoul National University (AIIS) and is also in charge of the Music and Audio Research Group (MARG) atSNU.

Lee claims in our exclusive and in-depth interview below that Supertone stands out in the AI audio landscape today because it is theoretically capable of creating an infinite number of new and original voices, as well as recreating existing voices.

This is made possible by Supertones foundation model called NANSY which stands for Neural Analysis & Synthesis and which Lee explains serves as the backbone of Supertones speech synthesis technologies. You can read the research paper for NANSY, here.

NANSY has the special ability to divide and re-assemble voice components timbre, linguistics, pitch, and loudness individually and independently, generating natural-sounding voices with unparalleled realism, he adds, (bolding MBWs).

Supertones AI vocal cloning tech first generated global media attention in January 2021 when it resurrected the voice of South Korean folk superstar Kim Kwang-seok to be played on Korean television show Competition of the Century: AI vs Human.

More recently, Supertone made headlines globally after recreating the voice of Kim Hyuk Gun, the vocalist of the popular Korean band The Cross, who was paralyzed in an accident. We collected 20 years of his voice data since debut and used it to train an AI voice in his unique vocal style, explains Lee.

HYBE also showcased the possibilities of what it can do with Supertones technology when it released a new singlecalledMasquerade from HYBE artist MIDNATT (aka Lee Hyun)last year. It was claimed byHYBE at the time to be the first-ever multilingual track produced in Korean, English, Japanese, Chinese, Spanish and Vietnamese.

In an increasingly global (yet localized) world, and amid the worldwide explosion of genres from K-Pop and J-Pop to Afrobeats and Spanish-language music, the opportunities presented by this use case of Supertones tech alone will likely have piqued the interest of music industry leaders worldwide.

Using this tech, a superstar artist think Taylor Swift, Billie Eilish or The Weeknd could release a new single in multiple languages in their actual voice on the same day.

According to Lee: Supertones multilingual pronunciation correction technology unlocks new avenues for artists to communicate with local fans in their native language, reaching out to the global market.

He adds: We hope this collaboration will establish a constructive precedent for AI technology supporting artists in overcoming language barriers to connect with global fans and broaden their musical spectrum.

MBW has previously asked if the companys newly acquired AI technology could ever be used to recreate the voices of superstar HYBE artists like BTS for projects that dont require the groups in-person participation, for example, while theyre serving in the military.

MBW readers who have been following our coverage of HYBEs financial performance over the years will recall that its Artist Indirect-Involvement business line revenue-generating projects that use an artists brand/likeness, without the actual artist needing to be involved became the companys primary revenue driver in 2020 in the absence of live shows during the pandemic.

In FY 2021, a year in which HYBE revenues surpassed $1 billion in revenues for the first time, the companys biggest organic revenue driver was, once gain, its Artist Indirect business, accounting for more than 60% of the companys revenues.

This Artist Indirect-Involvement business was only overtaken by the companys ArtistDirect Involvement business line in Q1 2022.

We asked Supertones President the same question: Will its tech ever be used to recreate the voices of superstars like BTS?

While Supertone is theoretically capable of creating an infinite number of new and original voices, as well as recreating existing voices, we are devoted to prioritizing the rights of all artists and creators, including those under HYBE.

Kyogu Lee

He tells us that, while Supertone is theoretically capable of creating an infinite number of new and original voices, as well as recreating existing voices, we are devoted to prioritizing the rights of all artists and creators, including those under HYBE.

He adds: Our focus with HYBE artists lies in facilitating seamless communication and interaction with global audiences, transcending all barriers, including language and geography.

Lee notes that HYBE is currently working on AI-dubbing some of its artists voices into foreign languages for parts of their video content, for example TOMORROW X TOGETHERs ACT: SWEET MIRAGE concert video, where the members comments were dubbed into Spanish using Supertones technology.

One of Supertones latest advancements is a real-time vocal changer called Supertone Shift that lets users switch between voices from a library of ten predefined voices. Users can then customize their chosen voice by adjusting the pitch, reverb and other effects.

Apart from the obvious production-related uses for this tool, the real-time capabilities could make it equally useful in a live setting. Just picture it: An artist could sing live on stage, and via multiple different AI-assisted voices, all switched in real-time.

Lee tells us that Supertone Shift has already hit 70,000 downloads and 30,000 monthly active users in just over two months since its beta launch.

The demand for expressing alter-egos has surged, adds Lee.

Beyond music, Lee says that he envisions Supertone Shift as the ultimate creative tool for a diverse range of content creators, including VTubers, livestreamers, podcasters, and gamers, enhancing the versatility and quality of their outputs.

HYBEs investment in Supertone arrived ahead of the current explosion of AI tech in the music industry and the wave of challenges it has brought with it. There are concerns about the source and legality of the training data used by many of the prominent AI music generators on the market today.

Music industry leaders have also raised the alarm about music streaming services and social media platforms being flooded with AI-generated songs. Some songwriters and artists, meanwhile, are worried about the threat of AI tools to their livelihoods.

According to Lee, AIs future contribution to the music industry will lie in expanding the creativity and imagination of creators and artists rather than replacing creators and creativity altogether.

Music, devoid of a storyteller the artist lacks the essential connection between storyteller (artist), story (music), and listener (fan), which leads me to believe that AI-generated music created without artist input may not endure, he says.

Meanwhile, for Supertone, Lee says that the HYBE subsidiary is focusing on evolving into a consumer-facing company this year, by offering what he calls artistic intelligence with its suite of AI tools for creators.

By providing convenient services that are universally accessible and applicable across diverse content fields, we aim to reduce creative barriers for professionals and individuals alike, says Lee.

Here, Supertones President and HYBEs resident AI expert, Kyogu Lee, tells us more about his companys tech, and his predictions for AI in the music business

Voices created through Supertones technology can be used in various areas, including acting and singing due to its rich expressions, which has reached new heights through our recent technological advancement to generate them in real-time. Moreover, fully equipped with our R&D lab, Content Business Development department, and in-house studio, Supertone transcends the scopes of a technology provider; it serves as a gateway to elevated content, offering new possibilities for content partners spanning music, broadcasting, movies, games, and beyond. We strive to add value to the content industry by amplifying creators artistic expression to produce more engaging content, and by introducing innovative voices to create new forms of content.

As we continue to collaborate with the creative industry, Supertones value is being appreciated across a wide range of content domains.

Notable achievements include our contribution to the Netflix series MASK GIRL released in August, 2023, where Supertones multi-speaker voice morphing technology brought to life the main character Kim Mo-mis alternative persona as an online streamer by producing a unique third voice from fusing voice tones of two actresses who played the character.

Additionally, in the Disney+ 2022 hit series Big Bet, Supertone utilized its voice De-aging technology, the industrys first attempt, to rejuvenate veteran actor Choi Min-siks voice for his character who was in his 30s. As we continue to collaborate with the creative industry, Supertones value is being appreciated across a wide range of content domains.

Kim Kwang-seok is a legendary singer cherished by Korean people with deep connections and affection, so we approached the project with utmost respect.

Although we were cautious given the unfamiliarity of voices created with SVS technology at that time, we had confidence in our ability to authentically resurrect his voice, leveraging Supertones forte in creating expressive voices that could deliver emotions and meanings through singing or speech.

Thankfully, the music industry and fans embraced the result with delight and gratitude. For the public, it provided a chance to observe new possibilities in the content realm, as AI reignited their nostalgia. Hearing Kims recreated voice, Kim Sang-wook, a prominent Korean scientist, responded, I hope this serves as an opportunity to explore AI and contemplate its coexistence with humanity. Im grateful it succeeded in its goal of evoking memories and ultimately resonating with fans as intended.

Supertone initially engaged with HYBE [formerly Big Hit Entertainment] in the first half of 2020. During this period, Supertones singing synthesis technology was gaining attention, and the late Kim Kwang-seoks project sparked interest from the entertainment industry, including HYBE, marking the beginning of our interaction.

HYBE had long been at the forefront of pioneering and advancing technological innovation in the entertainment sector.

HYBE had long been at the forefront of pioneering and advancing technological innovation in the entertainment sector. They recognized the promising trajectory of Supertones technology, including the innovative singing synthesis technology, which we both trusted would be suitable for the music industry. Concurrently, Supertone was firmly convinced of boundless possibilities and synergies that would arise from combining our technology with HYBEs global intellectual property (IP) and established production capabilities, which resulted in this partnership.

Acquired by HYBE in January 2023, Supertone contributes to HYBEs commitment to providing new avenues for content and fan experiences through solution businesses that leverage artists intellectual property (IP). Were currently in the process of running pilot projects across HYBEs various business areas, networks, and partnerships to advance Supertones technology and explore applications that can support and assist artists. Our technology can be utilized as a useful creative tool for some artists like MIDNATT who seek to experience new musical endeavors beyond technological limitations.

Supertone contributes to HYBEs commitment to providing new avenues for content and fan experiences through solution businesses that leverage artists intellectual property.

Additionally, it can enhance content immersion by integrating natural and expressive voice synthesis technology, as exemplified by Weverse Magazines Read-Aloud feature.

Were continuously discussing various business opportunities internally to innovate the possibilities of content creation.

MIDNATT project marks the first occasion where Supertone collaborated with an already existing artist to deliver more immersive and accessible music to fans worldwide. Following the release of his track Masquerade, we monitored a significant amount of positive responses from fans in various languages.

Some expressed how hearing their beloved artists in their native tongue and instantly comprehending the lyrics moved them like never before.

It was immensely gratifying and rewarding that they understood the intention and sincerity behind [the project].

Supertones extensive research into real-time AI voice conversion traces back to 2021, triggered by a conversation with an artist I met through a TV show. Despite being a beloved artist for a long time, he expressed regret over his voices inherent limitations in manifesting a wider range of expressions.

This made me realize that not only ordinary individuals like us but also those who captivate the public with beautiful voices desired to exert new vocal expressions.

Focusing on achieving real-time conversion of conversation-level voices, we showcased our initial project, then called NUVO, at the 2022 CES, where it won the Innovation Award. Later, we further refined the technology to a level suitable for live stages. This was demonstrated in 2023 when MIDNATT seamlessly transitioned between his vocal and a female vocal during a live performance. Achieving imperceptible latency prompted us to recognize the needs of real-time content creators, leading to the development of Supertone Shift.

We are fully aware of the controversy associated with AI technologies. Above all, whats crucial is to ensure that an artists creative intentions are conveyed, and that AI technologies are used as a catalyst for human creativity. It is our firm belief that we can only change perceptions by showcasing exemplary cases of how technology can assist artists and creators by collaborating closely with them. Creating meaningful content based on technology cannot happevn without inspiration and ideas that originate from creators.

We are fully aware of the controversy associated with AI technologies.

Recently, Supertone recreated the voice of Kim Hyuk Gun, the vocalist of the Korean band The Cross. After performing The Crosss music on a live stage together with the AI voice, Kim expressed his appreciation, saying, Thanks to the assistance of AI, he was able to successfully deliver a live performance despite his challenging conditions.

As showcased in this example, Supertone is constantly searching for ways to assist artists in overcoming creative barriers caused by physical or technological limitations.

However, we are often amazed by the innovative ideas and unexpected applications proposed by the artists and creators we collaborate with. Ultimately, I believe technology evolves in a mutually beneficial manner through ongoing interaction and engagement with artists and creators.

AI is being utilized throughout the entire process of creating, producing, distributing, and consuming music. Perhaps the most affected aspect of this is the creative process. However, I am personally skeptical if we can call this music produced solely from AI the evolution of music.

I am personally skeptical if we can call this music produced solely from AI the evolution of music.

To explain the reason behind this, we need to talk about the essence of music, which I believe is storytelling the fundamental purpose of all creative works and content.

Artists aspire to convey their intended story through the creative process, and various formats and genres of content have developed to maximize the effectiveness of their storytelling.

First and foremost, I believe establishing social consensus should be prioritized, one which will provide guidelines for identifying and addressing potential risks and issues caused by synthesized voices created without consent. This will mandate the AI industry to equip itself with the capability and readiness to respond to these issues.

We do not monetize on a voice without the permission of its rightful owner, under any circumstances.

Since its establishment, Supertone has adhered to the philosophy of developing products and conducting business in a manner that respects the intentions of creators. We also continue to enhance ethical guidelines and technological safeguards to prevent the abuse and misuse of AI technology. Supertone possesses watermark technology capable of detecting voices created by Supertone, and we have additionally initiated advanced research and development in this technology since April. In addition, we are actively cooperating to establish legal and institutional frameworks through continuous communication and interaction with relevant industries and policymakers. Throughout our endeavors, we will always prioritize the needs of creators and fans, striving to develop and apply relatable and coexistent technologies.

Supertone upholds the following three principles for responsible and ethical use of AI:

Supertone aspires to be the foremost choice by creators worldwide who seek solutions and services to produce voice content effectively and efficiently. We aim to imprint the equation #1 Voice AI Tech Provider = Supertone in the minds of all creators and potential customers globally.

As technology advances to facilitate music production and distribution, overproduction and oversaturation emerge as significant challenges.

The democratization of music production, fueled by advancements in creation and production technologies, has empowered numerous non-professionals to create music effortlessly.

As technology advances to facilitate music production and distribution, overproduction and oversaturation emerge as significant challenges.

Moreover, the widespread accessibility of the internet and various platforms has enabled global distribution of music.

This inundates listeners with an overwhelming amount of music on an increasingly larger scale, making it difficult for them to discover and explore music that aligns with their preferences. Addressing this challenge will require the development of systems or methodologies capable of identifying and delivering hidden, high-quality music to listeners.

View post:

Meet Kyogu Lee, President of Supertone - the voice cloning AI company acquired by HYBE for $32m - Music Business Worldwide

It's time to celebrate the incredible women leading the way in AI! Nominate your inspiring leaders for VentureBeats Women in AI Awards today before June 18. Learn More

Today, Dubai-based Camb AI, a startup researching AI-driven content localization technologies, announced the release of Mars5, a powerful AI model for voice cloning.

While there are plenty of models that can create digital voice replicas, including those from ElevenLabs, Camb claims to differentiate by offering a much higher level of realism with Mars5s outputs.

According to early samples shared by the company, the model not only emulates the original voice but also its complex prosodic parameters, including rhythm, emotion and intonation.

Camb also supports nearly 3 times as many languages as ElevenLabs: more than 140 languages compared to ElevenLabs 36, including low-resource ones like Icelandic and Swahili. However, the open-sourced technology, which can be accessed on GitHub starting today, is only the English-specific version. The version with expanded language support is available on the companys paid Studio.

VB Transform 2024 Registration is Open

Join enterprise leaders in San Francisco from July 9 to 11 for our flagship AI event. Connect with peers, explore the opportunities and challenges of Generative AI, and learn how to integrate AI applications into your industry. Register Now

The level of prosody and realism that Mars5 is able to capture, even with just a few seconds of input, is unprecedented. This is a mistral moment in speech, Akshat Prakash, the co-founder and CTO of the company, said in a statement.

Normally, voice cloning and text-to-speech conversion are two separate offerings. The former captures parameters from a given voice sample to create a voice clone while the latter uses that clone to convert any given text into synthetic speech. The technology, as we have seen in the past, has the potential to portray anyone as speaking anything.

With Mars5, Camb AI is taking the work ahead by mixing both capabilities into a unified platform. All a user has to do is upload an audio file, ranging between a few seconds to a minute, and provide the text content. The model will then use the speakers voice in the audio file as a reference, capture the relevant details including the original voice, speaking style, emotion, enunciation and meaning and synthesize the provided text as speech using it.

The company claims Mars5 can capture diverse emotional tones and pitches, covering all sorts of complex speech scenarios such as when a person is frustrated, commanding, calm or even spirited. This, Prakash noted, makes it suitable for content that has been traditionally difficult to convert into speech such as sports commentary, movies, and anime.

To achieve this level of prosody, Mars5 combines a Mistral-style ~750M parameter autoregressive model with a novel ~450M parameter non-autoregressive multinomial diffusion model, operating on 6kbps encodec tokens.

The AR model iteratively predicts the most coarse (lowest level) codebook value for the encodec features, while the NAR model takes the AR output and infers the remaining codebook values in a discrete denoising diffusion task. Specifically, the NAR model is trained as a DDPM using a multinomial distribution on encodec features, effectively inpainting the remaining codebook entries after the AR model has predicted the coarse codebook values, Prakash explained.

While specific benchmark stats are yet to be seen, early samples and tests (with a few seconds of reference audio) run by VentureBeat show that the model mostly performs better than popular open and closed-source speech synthesis models, including those from Metavoice and ElevenLabs. The competitive offerings synthesized speech clearly but the results didnt sound as similar to the original voice as they did in the case of Mars5.

ElevenLabs is closed source so its hard to specifically say why they arent able to capture nuances that we can, but given that they report training on 500K+ hours (almost 5 times the dataset we have in English), it is clear to us that we have a superior model design that learns speech and its nuances better than theirs. Of course, as our datasets continue to grow and Mars5 trains even more, which we will release in successive checkpoints in Github, we expect it to only get better and better and better, especially considering support from the open-source community, the CTO added.

As the company continues to bolster the voice cloning and text-to-speech performance of Mars5, it is also planning the open-source release of another model called Boli. This one has been designed to enable translation with contextual understanding, correct grammar and apt colloquialism.

Boli is our proprietary translation model, which surpasses traditional engines such as Google Translate and DeepL in capturing the nuances and colloquial aspects of language. Unlike large-scale parallel corpus-based systems, Boli offers a more consistent and natural translation experience, particularly in low- to medium-resource languages. Feedback from clients indicates that Bolis translations outperform those produced by mainstream tools, including the latest generative models like ChatGPT, Prakash said.

Currently, both Mars5 and Boli work with 140 languages on the Cambs proprietary platform Camb Studio. The company is also providing these capabilities as APIs to enterprises, SMEs and developers. Prakash did not share the exact number of customers but he did point out the company is working with Major League Soccer, Tennis Australia, Maple Leaf Sports & Entertainment as well as leading movie and music studios and several government agencies.

For Major League Soccer, Camb AI live-dubbed a game into four languages in parallel for over 2 hours, uninterrupted becoming the first company to do so. It also translated the Australian Opens post-match conference into multiple languages and translated the psychological thriller Three from Arabic to Mandarin.

VB Daily

Stay in the know! Get the latest news in your inbox daily

By subscribing, you agree to VentureBeat's Terms of Service.

Thanks for subscribing. Check out more VB newsletters here.

An error occured.

Go here to read the rest:

Exclusive: Camb takes on ElevenLabs with open voice cloning AI model Mars5 offering higher realism, support for 140 ... - VentureBeat

Cell Microsystems technologies allow researchers to image, identify and isolate viable single cells for analysis more successfully and efficiently than ever. Its coreCellRaft technologywas invented in the UNC-Chapel Hill lab of Dr.Nancy Allbritton, who co-founded the company in 2010 with chemistry professorDr. Christopher Sims and researcherYuli Wang in 2010.

Scientists working in the pharma-biotech and academic industries need to isolate single cells to understand and develop treatments for diseases. The traditional methods they relied on in the past, such as single-cell RNA sequencing, can destroy the original cell, are labor- and time-intensive and have low yield rates.

The CellRaft Array technology created at Carolina allows scientists to get better results faster. The traditional way of doing single-cell clonal propagation is a 10-week process, but with our CellRaft Array, you can go from single cells to a plate full of clones in five to 10 days, said Gary Pace, who became the Cell MicrosystemsCEO in 2014.

Over the past 10 years, Cell Microsystems tested its technology and developed products based on the needs of its customers. One of the companys early breakthroughs came through CRISPR, technology that works like molecular scissors to cut DNA at specific locations and help scientists add, remove or replace genetic material to treat genetic diseases.

We played with other applications before that, but they didnt address a particular market, Pace said. Once we recognized CRISPR as a key application, we saw there was a whole world out there built around clonal propagation from single cells a world that has only gotten bigger. CRISPR allowed us to begin to focus on specific markets.

To extend the power of its core technology, the company developed anautomated platform that allows scientists to watch a single cell divide multiple times. Captured images help researchers identify specific cell attributes that are important for further analysis.

The platform is driven by Cell Microsystems proprietary software calledCellRaft Cytometry, which automates how the system isolates cells and captures images. The softwares image-based verification capabilities let researchers specify precise attributes theyre interested in such as cell shape or colony size and then identify cell colonies that meet their specifications.

The software brings together insights on single-cell propagation, clone colony size and observable genetic characteristics. Our CellRaft Cytometry software creates a Venn diagram of those three distinct observations, and then you can isolate just the cells in the position on the array that overlap all three of those circles of the Venn diagram, Pace said. Weve collapsed a number of the different modalities of a single-cell workflow into a single platform. And thats very powerful. No one else has that.

Cell Microsystems licenses intellectual property for the foundational technology invented at UNC-Chapel Hill used in the CellRaft product and worked with theUNC Office of Technology Commercialization on other joint patents for the companys automated platform. The company has also filed patents on its own inventions.

With a well-integrated set of technologies built around user needs, Cell Microsystems offers researchers a single solution that packs a powerful punch. For scientists, there are benefits across the board: high viability, very efficient, amenable to a large number of cell lines and an integrated platform that gives you cytometric data that you cant get anywhere else, Pace said.

Read more about Cell Microsystems.

Visit link:

Single-cell cloning solution speeds breakthroughs | UNC-Chapel Hill - The University of North Carolina at Chapel Hill