Meet Kyogu Lee, President of Supertone – the voice cloning AI company acquired by HYBE for $32m – Music Business Worldwide

MBWs World Leaders is a regular series in which we turn the spotlight toward some of the most influential industry figures overseeing key international markets. In this feature, we speak to Kyogu Lee, President of HYBE-owned voice AI company Supertone. World Leaders is supported byPPL.

AI technology is a big priority for HYBE, the South Korean entertainment giant behind K-pop superstars BTS.

Evidence of that arrived in March when HYBEs CEO Jiwon Park consented to have his own voice cloned to demonstrate the capabilities of the companys proprietary AI on its Q1 investor call.

HYBEs so-called voice synthesis technology was developed by Supertone, the AI voice replication software startup in which HYBE acquired a majority stake in a $32 million deal in 2023.

Founded in Seoul in 2020, Supertone claims to be able to create a hyper-realistic and expressive voice that [is not] distinguishable from real humans.

Supertones purported ability to do just that makes the apparent strategy behind HYBEs multi-million dollar investment in the technology a lot clearer, when viewed through the lens of comments shared by HYBE Chairman Bang Si-Hyuk in an interview with Billboardlast year.

I have long doubted that the entities that create and produce music will remain human, saidBang Si-Hyuk.

I dont know how long human artists can be the only ones to satisfy human needs and human tastes. And thats becoming a key factor for my operation and a strategy for HYBE.

By acquiring Supertone, HYBE has also brought into the fold the startups co-founder and President Kyogu Lee, a widely respected AI expert with a PhD in Computer-Based Music Theory and Acoustics from Stanford University.

In addition to heading up Supertone at HYBE, he leads applied research at the Artificial Intelligence Institute at Seoul National University (AIIS) and is also in charge of the Music and Audio Research Group (MARG) atSNU.

Lee claims in our exclusive and in-depth interview below that Supertone stands out in the AI audio landscape today because it is theoretically capable of creating an infinite number of new and original voices, as well as recreating existing voices.

This is made possible by Supertones foundation model called NANSY which stands for Neural Analysis & Synthesis and which Lee explains serves as the backbone of Supertones speech synthesis technologies. You can read the research paper for NANSY, here.

NANSY has the special ability to divide and re-assemble voice components timbre, linguistics, pitch, and loudness individually and independently, generating natural-sounding voices with unparalleled realism, he adds, (bolding MBWs).

Supertones AI vocal cloning tech first generated global media attention in January 2021 when it resurrected the voice of South Korean folk superstar Kim Kwang-seok to be played on Korean television show Competition of the Century: AI vs Human.

More recently, Supertone made headlines globally after recreating the voice of Kim Hyuk Gun, the vocalist of the popular Korean band The Cross, who was paralyzed in an accident. We collected 20 years of his voice data since debut and used it to train an AI voice in his unique vocal style, explains Lee.

HYBE also showcased the possibilities of what it can do with Supertones technology when it released a new singlecalledMasquerade from HYBE artist MIDNATT (aka Lee Hyun)last year. It was claimed byHYBE at the time to be the first-ever multilingual track produced in Korean, English, Japanese, Chinese, Spanish and Vietnamese.

In an increasingly global (yet localized) world, and amid the worldwide explosion of genres from K-Pop and J-Pop to Afrobeats and Spanish-language music, the opportunities presented by this use case of Supertones tech alone will likely have piqued the interest of music industry leaders worldwide.

Using this tech, a superstar artist think Taylor Swift, Billie Eilish or The Weeknd could release a new single in multiple languages in their actual voice on the same day.

According to Lee: Supertones multilingual pronunciation correction technology unlocks new avenues for artists to communicate with local fans in their native language, reaching out to the global market.

He adds: We hope this collaboration will establish a constructive precedent for AI technology supporting artists in overcoming language barriers to connect with global fans and broaden their musical spectrum.

MBW has previously asked if the companys newly acquired AI technology could ever be used to recreate the voices of superstar HYBE artists like BTS for projects that dont require the groups in-person participation, for example, while theyre serving in the military.

MBW readers who have been following our coverage of HYBEs financial performance over the years will recall that its Artist Indirect-Involvement business line revenue-generating projects that use an artists brand/likeness, without the actual artist needing to be involved became the companys primary revenue driver in 2020 in the absence of live shows during the pandemic.

In FY 2021, a year in which HYBE revenues surpassed $1 billion in revenues for the first time, the companys biggest organic revenue driver was, once gain, its Artist Indirect business, accounting for more than 60% of the companys revenues.

This Artist Indirect-Involvement business was only overtaken by the companys ArtistDirect Involvement business line in Q1 2022.

We asked Supertones President the same question: Will its tech ever be used to recreate the voices of superstars like BTS?

While Supertone is theoretically capable of creating an infinite number of new and original voices, as well as recreating existing voices, we are devoted to prioritizing the rights of all artists and creators, including those under HYBE.

Kyogu Lee

He tells us that, while Supertone is theoretically capable of creating an infinite number of new and original voices, as well as recreating existing voices, we are devoted to prioritizing the rights of all artists and creators, including those under HYBE.

He adds: Our focus with HYBE artists lies in facilitating seamless communication and interaction with global audiences, transcending all barriers, including language and geography.

Lee notes that HYBE is currently working on AI-dubbing some of its artists voices into foreign languages for parts of their video content, for example TOMORROW X TOGETHERs ACT: SWEET MIRAGE concert video, where the members comments were dubbed into Spanish using Supertones technology.

One of Supertones latest advancements is a real-time vocal changer called Supertone Shift that lets users switch between voices from a library of ten predefined voices. Users can then customize their chosen voice by adjusting the pitch, reverb and other effects.

Apart from the obvious production-related uses for this tool, the real-time capabilities could make it equally useful in a live setting. Just picture it: An artist could sing live on stage, and via multiple different AI-assisted voices, all switched in real-time.

Lee tells us that Supertone Shift has already hit 70,000 downloads and 30,000 monthly active users in just over two months since its beta launch.

The demand for expressing alter-egos has surged, adds Lee.

Beyond music, Lee says that he envisions Supertone Shift as the ultimate creative tool for a diverse range of content creators, including VTubers, livestreamers, podcasters, and gamers, enhancing the versatility and quality of their outputs.

HYBEs investment in Supertone arrived ahead of the current explosion of AI tech in the music industry and the wave of challenges it has brought with it. There are concerns about the source and legality of the training data used by many of the prominent AI music generators on the market today.

Music industry leaders have also raised the alarm about music streaming services and social media platforms being flooded with AI-generated songs. Some songwriters and artists, meanwhile, are worried about the threat of AI tools to their livelihoods.

According to Lee, AIs future contribution to the music industry will lie in expanding the creativity and imagination of creators and artists rather than replacing creators and creativity altogether.

Music, devoid of a storyteller the artist lacks the essential connection between storyteller (artist), story (music), and listener (fan), which leads me to believe that AI-generated music created without artist input may not endure, he says.

Meanwhile, for Supertone, Lee says that the HYBE subsidiary is focusing on evolving into a consumer-facing company this year, by offering what he calls artistic intelligence with its suite of AI tools for creators.

By providing convenient services that are universally accessible and applicable across diverse content fields, we aim to reduce creative barriers for professionals and individuals alike, says Lee.

Here, Supertones President and HYBEs resident AI expert, Kyogu Lee, tells us more about his companys tech, and his predictions for AI in the music business

Voices created through Supertones technology can be used in various areas, including acting and singing due to its rich expressions, which has reached new heights through our recent technological advancement to generate them in real-time. Moreover, fully equipped with our R&D lab, Content Business Development department, and in-house studio, Supertone transcends the scopes of a technology provider; it serves as a gateway to elevated content, offering new possibilities for content partners spanning music, broadcasting, movies, games, and beyond. We strive to add value to the content industry by amplifying creators artistic expression to produce more engaging content, and by introducing innovative voices to create new forms of content.

As we continue to collaborate with the creative industry, Supertones value is being appreciated across a wide range of content domains.

Notable achievements include our contribution to the Netflix series MASK GIRL released in August, 2023, where Supertones multi-speaker voice morphing technology brought to life the main character Kim Mo-mis alternative persona as an online streamer by producing a unique third voice from fusing voice tones of two actresses who played the character.

Additionally, in the Disney+ 2022 hit series Big Bet, Supertone utilized its voice De-aging technology, the industrys first attempt, to rejuvenate veteran actor Choi Min-siks voice for his character who was in his 30s. As we continue to collaborate with the creative industry, Supertones value is being appreciated across a wide range of content domains.

Kim Kwang-seok is a legendary singer cherished by Korean people with deep connections and affection, so we approached the project with utmost respect.

Although we were cautious given the unfamiliarity of voices created with SVS technology at that time, we had confidence in our ability to authentically resurrect his voice, leveraging Supertones forte in creating expressive voices that could deliver emotions and meanings through singing or speech.

Thankfully, the music industry and fans embraced the result with delight and gratitude. For the public, it provided a chance to observe new possibilities in the content realm, as AI reignited their nostalgia. Hearing Kims recreated voice, Kim Sang-wook, a prominent Korean scientist, responded, I hope this serves as an opportunity to explore AI and contemplate its coexistence with humanity. Im grateful it succeeded in its goal of evoking memories and ultimately resonating with fans as intended.

Supertone initially engaged with HYBE [formerly Big Hit Entertainment] in the first half of 2020. During this period, Supertones singing synthesis technology was gaining attention, and the late Kim Kwang-seoks project sparked interest from the entertainment industry, including HYBE, marking the beginning of our interaction.

HYBE had long been at the forefront of pioneering and advancing technological innovation in the entertainment sector.

HYBE had long been at the forefront of pioneering and advancing technological innovation in the entertainment sector. They recognized the promising trajectory of Supertones technology, including the innovative singing synthesis technology, which we both trusted would be suitable for the music industry. Concurrently, Supertone was firmly convinced of boundless possibilities and synergies that would arise from combining our technology with HYBEs global intellectual property (IP) and established production capabilities, which resulted in this partnership.

Acquired by HYBE in January 2023, Supertone contributes to HYBEs commitment to providing new avenues for content and fan experiences through solution businesses that leverage artists intellectual property (IP). Were currently in the process of running pilot projects across HYBEs various business areas, networks, and partnerships to advance Supertones technology and explore applications that can support and assist artists. Our technology can be utilized as a useful creative tool for some artists like MIDNATT who seek to experience new musical endeavors beyond technological limitations.

Supertone contributes to HYBEs commitment to providing new avenues for content and fan experiences through solution businesses that leverage artists intellectual property.

Additionally, it can enhance content immersion by integrating natural and expressive voice synthesis technology, as exemplified by Weverse Magazines Read-Aloud feature.

Were continuously discussing various business opportunities internally to innovate the possibilities of content creation.

MIDNATT project marks the first occasion where Supertone collaborated with an already existing artist to deliver more immersive and accessible music to fans worldwide. Following the release of his track Masquerade, we monitored a significant amount of positive responses from fans in various languages.

Some expressed how hearing their beloved artists in their native tongue and instantly comprehending the lyrics moved them like never before.

It was immensely gratifying and rewarding that they understood the intention and sincerity behind [the project].

Supertones extensive research into real-time AI voice conversion traces back to 2021, triggered by a conversation with an artist I met through a TV show. Despite being a beloved artist for a long time, he expressed regret over his voices inherent limitations in manifesting a wider range of expressions.

This made me realize that not only ordinary individuals like us but also those who captivate the public with beautiful voices desired to exert new vocal expressions.

Focusing on achieving real-time conversion of conversation-level voices, we showcased our initial project, then called NUVO, at the 2022 CES, where it won the Innovation Award. Later, we further refined the technology to a level suitable for live stages. This was demonstrated in 2023 when MIDNATT seamlessly transitioned between his vocal and a female vocal during a live performance. Achieving imperceptible latency prompted us to recognize the needs of real-time content creators, leading to the development of Supertone Shift.

We are fully aware of the controversy associated with AI technologies. Above all, whats crucial is to ensure that an artists creative intentions are conveyed, and that AI technologies are used as a catalyst for human creativity. It is our firm belief that we can only change perceptions by showcasing exemplary cases of how technology can assist artists and creators by collaborating closely with them. Creating meaningful content based on technology cannot happevn without inspiration and ideas that originate from creators.

We are fully aware of the controversy associated with AI technologies.

Recently, Supertone recreated the voice of Kim Hyuk Gun, the vocalist of the Korean band The Cross. After performing The Crosss music on a live stage together with the AI voice, Kim expressed his appreciation, saying, Thanks to the assistance of AI, he was able to successfully deliver a live performance despite his challenging conditions.

As showcased in this example, Supertone is constantly searching for ways to assist artists in overcoming creative barriers caused by physical or technological limitations.

However, we are often amazed by the innovative ideas and unexpected applications proposed by the artists and creators we collaborate with. Ultimately, I believe technology evolves in a mutually beneficial manner through ongoing interaction and engagement with artists and creators.

AI is being utilized throughout the entire process of creating, producing, distributing, and consuming music. Perhaps the most affected aspect of this is the creative process. However, I am personally skeptical if we can call this music produced solely from AI the evolution of music.

I am personally skeptical if we can call this music produced solely from AI the evolution of music.

To explain the reason behind this, we need to talk about the essence of music, which I believe is storytelling the fundamental purpose of all creative works and content.

Artists aspire to convey their intended story through the creative process, and various formats and genres of content have developed to maximize the effectiveness of their storytelling.

First and foremost, I believe establishing social consensus should be prioritized, one which will provide guidelines for identifying and addressing potential risks and issues caused by synthesized voices created without consent. This will mandate the AI industry to equip itself with the capability and readiness to respond to these issues.

We do not monetize on a voice without the permission of its rightful owner, under any circumstances.

Since its establishment, Supertone has adhered to the philosophy of developing products and conducting business in a manner that respects the intentions of creators. We also continue to enhance ethical guidelines and technological safeguards to prevent the abuse and misuse of AI technology. Supertone possesses watermark technology capable of detecting voices created by Supertone, and we have additionally initiated advanced research and development in this technology since April. In addition, we are actively cooperating to establish legal and institutional frameworks through continuous communication and interaction with relevant industries and policymakers. Throughout our endeavors, we will always prioritize the needs of creators and fans, striving to develop and apply relatable and coexistent technologies.

Supertone upholds the following three principles for responsible and ethical use of AI:

Supertone aspires to be the foremost choice by creators worldwide who seek solutions and services to produce voice content effectively and efficiently. We aim to imprint the equation #1 Voice AI Tech Provider = Supertone in the minds of all creators and potential customers globally.

As technology advances to facilitate music production and distribution, overproduction and oversaturation emerge as significant challenges.

The democratization of music production, fueled by advancements in creation and production technologies, has empowered numerous non-professionals to create music effortlessly.

As technology advances to facilitate music production and distribution, overproduction and oversaturation emerge as significant challenges.

Moreover, the widespread accessibility of the internet and various platforms has enabled global distribution of music.

This inundates listeners with an overwhelming amount of music on an increasingly larger scale, making it difficult for them to discover and explore music that aligns with their preferences. Addressing this challenge will require the development of systems or methodologies capable of identifying and delivering hidden, high-quality music to listeners.

View post:

Meet Kyogu Lee, President of Supertone - the voice cloning AI company acquired by HYBE for $32m - Music Business Worldwide