AI SETS NEW MUSICAL BOUNDARIES

Before erhu player Wang Chuting started to perform an original piece of music titled Over the Rainbow, Zhang Yuan, who composed the work, stepped onto the stage and sat next to Wang.

They both donned brainwave-reading headsets, signaling that the performance had officially started. The headsets detect signals from the brains of the performer and composer, transforming them into music to allow real-time improvisation.

In addition to the erhu, Wang played guitar in the piece, which is not related to the eponymous song featured in the movie The Wizard of Oz.

Zhang, who graduated from the Central Conservatory of Music in Beijing with a bachelor's degree and a master's in composition, said, "The interactive element highlighted the dynamic connection between the musicians, the audience and the AI (artificial intelligence) technology, blurring the lines between human expression and machine capabilities."

The composer holds a doctorate in electronic music composition from the Conservatory's music AI and information technology department.

"Traditionally, performers play onstage and the audience listens. The communication is silent most of the time. However, music is now listening to us for a change. Performers can see emotional changes in the listeners, which affect performances and the way in which musicians play," Zhang said.

He added that brainwave-reading headsets are often used in medical treatment.

Zhang's work was staged recently at a groundbreaking Central Conservatory of Music concert that combined AI technology and human musicians, redefining the traditional concert experience.

Before entering the concert hall, the audience was given a taste of the futuristic elements of music through a compositional system generated by AI technology. By scanning a code, and texting the lyrics, rhythm and vocal timbre, the audience could obtain a song totally created by AI within 23 seconds.

Li Xiaobing, executive director of the concert and head of the music AI and information technology department at the Conservatory, said: "AI is everywhere and is radically changing our daily lives, for example, in shopping, commuting and banking. In music, AI is fundamentally changing not just how we listen to it, but how music is created and even how the music industry operates.

"There is a big question for musicians: is AI going to assist them, or is it going to be a musician? This concert, with the theme 'music from the future', might offer a clue.

"With AI's assistance, music creators can analyze real-time performance data and empower human performers to make adjustments based on audience feedback, ultimately enhancing the overall concert experience."

Li added that the music AI and information technology department was founded at the Conservatory in 2019. The department comprises an AI music center, an electronic music composition center and music therapy center. The recent concert was the first time the department had showcased its doctorate holders' musical works to the public.

New experiences

Throughout the performance, the constant mix of tradition and innovation pushed the boundaries of musical possibilities, with the audience switching its attention between human involvement and the use of AI technology.

Li said: "We are showing people the evolving role of this technology in artistic expression. As AI continues to advance, it has become an indispensable tool for musicians, allowing them to create new sounds, understand audiences' emotional reactions, and ultimately break new ground in terms of musical experiences."

Born and raised in Nanchang, Jiangxi province, Li studied piano and flute as a child. When he was a teenager, he developed an interest in electronic music, including sequencing, synthesis and sampling.

With the rapid development of computer technology, and with AI technology making its mark in music, Li went to Hong Kong in the early 2000s to learn the latest electronic music and interactive music technology. At that time, he also taught at the Xinghai Conservatory of Music in Guangzhou, capital of Guangdong province.

Later, he studied with Chinese composer Wu Zuqiang (1927-2022) and obtained a doctorate in composition from the Central Conservatory of Music in 2011.

The recent concert also featured three works by composer Luan Jia, who holds a doctorate in electronic music composition from the Central Conservatory of Music. One of his works, Er No. 2 — Si Mgang, combined AI and traditional instruments played by members of the Va ethnic group from Yunnan province.

In the Va group's language, si mgang translates as "caves" — a reference to humans living in caves in ancient times.

Xu Shenyang, a student at the Conservatory, played three instruments used by the Va ethnic group: the de (a type of horn), the ling (bell) and the kou xian (Jew's Harp).Based on motion sensing control, which reshapes the sound of a live performance in real time, the ancient instruments could be heard in an immersive 3D format.

Luan, who has composed for dance dramas, operas and musicals, said: "I was curious about the de, which is made from wood or cow's horn in Yunnan. I once listened to an album featuring this instrument, which produces a sound very similar to that which we use in electronic music composition. It's very interesting exploring sounds from the past and from the future."

Breakthroughs in AI technology have triggered anxiety among professional musicians, producers, engineers and others in the music industry, who fear their livelihoods could be threatened.

Luan said that with the help of AI technology, it is no longer necessary to have musical knowledge or training, as anyone can potentially create a hit song with the assistance of computers that evolve with each artificially produced guitar lick or drumbeat.

"However, in pursuing the best technology, we tend not to think about the impact AI can have on real musicians who create real art. The human element will never be replaced," Luan added.

Concert highlights

Other highlights of the concert included Starry Night, a composition by Sun Yuming which showcased AI's deep learning ability to capture guzheng performers' gestures and expressions. Drizzle Path, composed by Zhao Yixuan, highlighted the use of AI algorithms to generate new timbres.

"From the start, it was always clear to us that we wanted to approach AI as an assistive technology, not a replacement technology. If AI is used merely as a tool by human artists, it can be seen as just another technological progression in music creation," Zhao said.

A composer and synthesizer musician, whose compositions mainly focus on electroacoustic, interactive and contemporary music, Zhao is currently a post-doctoral researcher at the Central Conservatory of Music.

Li, from the Conservatory, said the institution has launched its AI technology to create children's songs.

In June last year, Kugou, an online music streaming platform owned by Tencent Entertainment Group, introduced its first AI singer.

By learning to reproduce singer-actress Yang Chaoyue's voice, the AI performer not only boasts singing ability, but also imitates human emotions to sing like a real person. Kugou used the technology to release new songs performed by the AI singer imitating pop stars such as Teresa Teng (1953-95) and Anita Mui (1963-2003).

On June 15, NetEase Cloud Music, a mainstream online streaming platform, announced its latest AI singer technology, which generates highly expressive singing voices. A virtual music label, WOWAIDO, was also launched, featuring 12 virtual singers and three new songs.

Discussion of AI technology intensified in May, when Singaporean pop singer Stefanie Sun became a hot topic on Chinese social media platforms due to an AI version of her voice.

More than 1,000 music videos featuring Sun's "voice" created a buzz after they appeared on the popular Chinese video-sharing platform Bilibili. In addition to "singing" in Sun's voice, the AI software replicated the voices of other singers, including Chinese pop star Faye Wong and Singaporean singer-songwriter JJ Lin.

As well as those who are either amazed by the AI technology or claim it is a form of "cheating", legal professionals have voiced concerns about the possibility of rights infringements.

Li said: "Technology has been transforming music, and art broadly, for generations. For example, when MIDI (Musical Instrument Digital Interface) appeared in the 1980s, there were numerous discussions about it, similar to those today about AI technology. Such discussions will never end, due to the rapid emergence of more new technologies. Musicians are already using AI to their advantage in creative ways.

"The power rests with the person using the technology. Artists still make choices about style and instrumentation. They can edit, cut and paste, delete and play with the 'stems' the AI generates. They are still the creators, and are just using a modern technology for ideas."