AI meets symphony in futuristic blend in NY

As the conductor guided the orchestra's musicians with precision, he suddenly reached for a laptop, tapping commands that synced the digital with the symphonic.

That nuance stood out as the Orchestra Now from Bard College in Annandale-on-Hudson, New York state, performed at the Stern Auditorium at Carnegie Hall on Saturday to open the seventh annual China Now Music Festival in New York City with the theme Composing the Future.

At the weeklong festival that runs through Saturday, different generations of Chinese composers from the 1960s to the present will feature pieces that merge traditional elements with emerging music techniques, including the use of artificial intelligence.

Sun Yuming, a composer, music producer and lecturer on electronic music composition at the Central Conservatory of Music in Beijing, introduced his piece Starry Night to the New York audience.

Instead of offering a composer's traditional acknowledgment after the symphony while seated in the audience, Sun took to the stage with his laptop, programing in real time with conductor Cai Jindong, who was also equipped with a laptop and the full musical score.

The composition featured AI-driven visual algorithms and real-time timbre rendering to blend physical and virtual instruments.

"Sound has countless forms of expression, and through technological means, we create simulated sounds, as if crafting a virtual instrument that fuses electronic and traditional music," Sun said.

"This approach combines the unique characteristics of traditional instruments with the innovations of electronic music, integrating AI throughout the entire performance — not only in sound but also in the visual effects on stage."

The collaboration between AI programing and musical composition brings a new dimension, allowing AI to be woven throughout the creative process — from brainstorming and concept development to arranging instrumentation, refining and editing, he said.

Sun's "instruments" — his laptop and mouse — allowed him to precisely control pitch, note duration, rhythm and sound location simultaneously.

And when Cai — director of the US-China Music Institute at Bard College Conservatory of Music — raised his hand to various levels, AI technology was used to simulate different instrument sounds, interpreting the movements through skeletal recognition software.

The Central Conservatory of Music also created an Artificial Intelligence Composition System, featuring two pieces: Overture: Welcome and A Thousand Miles of Rivers and Mountains.

At a preconcert panel discussion, Li Xiaobing, director of the Department of Music Artificial Intelligence and Music Information Technology at the conservatory, told the media, "Creative endeavors drive the evolution of art, and artificial intelligence similarly propels creativity forward."

Li said he would try to bring a "robot conductor" to New York so that it could not only keep up with the speed and rhythm of human performers but also provide real-time cues and even imitate the styles of different conductors.

'Ideal state'

"We don't know what the future holds for AI technology with music — it might reach an ideal state or even surpass our expectations," he said.

Cai said he expects more music and art exchanges between China and the United States.

"We have such AI technology, and we want more and more talented Chinese composers to present their work to larger audiences," he said.

Anthony Paul De Ritis, a composer, professor and former chair of the Music Department at Northeastern University in Boston, told China Daily that dialogue between the two countries on music is needed.

He has been working as a scholar in China since 1999 and exploring music-making with traditional Chinese instruments, featuring electric or electric acoustic effects based on Chinese poems.

"The exchange of ideas over this 25-year period has been incredibly beneficial for me. Music is a means of intercultural dialogue," he said.

Ritis is also the research coordinator of the International Computer Music Association in California, where he encourages more connections through music.

"China is very much supportive of the exchanges of these academic ideas," he said, which "is going to cross multiple cultures".