AI and the future of movies

The use of artificial intelligence in filmmaking has become a global focal point, taking center stage at the recently concluded 28th Shanghai International Film Festival, China's only A-list film festival accredited by the International Federation of Film Producers Associations.

At the SIFForum held in mid-June, the festival's premier platform for industry discourse, AI was a dominant subject. Industry insiders, optimistic or skeptical, agreed that the fast-evolving technology is bringing drastic changes to the industry.

From script generators and automated editing tools to trailer generators, industry insiders observed that AI is extensively permeating visual creation, reshaping how stories are told, visually presented, and marketed.

In China, AI-generated micro-dramas are rapidly gaining traction, and AI technology is already playing a pivotal role in the postproduction of feature films.

Yet there are still reservations and mistrust about the role of AI in filmmaking. Some, such as Quji Xiaojiang, head of Bona Film Group's AIGMS production center, were worried that AI might strip films of their long-cherished human emotional touch.

Her sentiments were echoed by Tony Leung Chiu-wai, president of the jury for the Golden Goblet Awards, the top honor of the SIFF. Leung pointed out that AI operates strictly within human-defined parameters and existing reference materials, so it "has no soul".

Speaking at the SIFForum session Smart Tech, Immersive Worlds: The Next Film Revolution, Quji emphasized the gap between short-form AI content and cinematic standards, saying that "the current technology still can't meet the public's expectations for the standards of high-quality movies".

Although some AI-generated content claims to be movie-level quality, she said, "AI-generated short videos on mobile phones and cinema-level feature films are fundamentally different in terms of storytelling and production.

"A movie screen is over 8,000 times bigger than a regular phone. Just because you find a great AI-generated short video on your phone doesn't mean you can sit through two hours of it in a movie theater."

Wang Changtian, chairman of Beijing Enlight Media, an entertainment conglomerate and producer of a series of hit TV shows and movies, also questioned the power of AI in reshaping the filmmaking industry.

"I think at the middle to lower levels, for basic entertainment products and content, AI can probably get more involved," he said. "But for high-end content, where we demand a masterwork that brings together all kinds of artistic forms, I think technology is just one of many factors."

Eventually, it is always the people — those involved in the creation, and the public demand for visual products — that actually affect the industry, he added.

To address these industry concerns, the SIFF launched AI Backlot, a pioneering initiative pairing AI creators with filmmakers to collaborate on short films. The new project was "not just a simple AI short video contest, nor is it a creative challenge", said Tong Ying, deputy director of the Shanghai International Film & TV Festival Center. Rather, the center wanted to hold a blended experiment, observe the process and shed light on the possibilities of AI use in future moviemaking.

"We focused not only on the final work, but also on everything that happened during the creation of an AI film: how ideas are generated, how tasks are divided, how traditional experiences such as directing, scriptwriting, and cinematography are incorporated into the new creative process, and what problems still require human judgment and experience to solve."

The initiative received about 500 applications from seven countries and regions. After the initial selection, 22 outstanding creators made it to the finals. In Shanghai, following presentations and evaluations, four teams, each consisting of a filmmaker and an AI creator, participated in a monthlong collaborative film creation.

The fruits of their joint work — a five-minute short film by each team — were released during the 28th SIFF, and the AI Backlot workshop was open to the public from June 14 to 15, when the creators showed the public their working methods, processes and ideas.

The four teams "each explored a long way in their distinctive direction", Tong said.

The team Bicycle Kids, consisting of Chinese filmmaker Hou Zuxin and German AI creator Mark Wachholz, won the Honor for Industrial Exploration with their creation of a philosophical story about a butterfly's metamorphosis.

AI has been a powerful tool, helping the team create images for abstract and complex ideas — images that people of different cultural backgrounds can connect to and resonate with, Wachholz told China Daily after the premiere. A veteran of AI, Wachholz welcomes the model's uncontrollable outputs, viewing them as creative inspiration rather than enforcing strict directorial control.

In one scene, he wanted to create a disturbing and intense clip of the caterpillar turning into the butterfly. "We used the prompt, something like a 'horror documentary'," he said. By feeding these two words to AI, they managed to arrive at something in between and achieve the visual effects they were searching for. "This is a new kind of language, or a new kind of thinking, how to talk to these AI models," Wachholz said.

Having taken part in AI video contests all over the world, Wachholz said there are powerful video models in the United States and in China, but Europe doesn't yet have them.

Currently, AI models are developing at such a rapid pace that "it is a horse race … one may be better for two weeks, until another competitor comes up", he said. "But right now, at this point in time, the Chinese video models are better, and everyone knows this."

He has found that in China, there seems to be "a deliberate setup to have multiple competitors" in the sector. "You have Kling, MiniMax, and Seedance (ByteDance's latest AI video generation model), and Alibaba has also jumped into the race. This creates pressure to be even better and better."

Forty-five-year-old film director Huang Lei and Wu Hankun, a young actor and AI creator, formed the team Lightcone. Huang confessed that he was pressured to get involved with AI because he has had colleagues who have had difficulty getting their film projects made in the past few years.

"When you don't have adequate funding or the resources to make a feature film, at least AI can help you construct a preliminary outlook of what you want to show. That's very useful in the fundraising stage of movie production.

"Very often we found AI actors' performances stiff and unrealistic, so we decided to take advantage of Hankun's background as a professional actor," Huang added. "We shot his performances first, then used AI to transform his appearance for different characters, while retaining the authentic details of human acting."

But that didn't completely solve the problem. "AI tends to smooth out performance imperfections, such as micro-expressions and vocal cracks," Wu said. "But these are exactly what make your acting moving, personal, and real."

Also, purely AI-generated performances suffer from inconsistency and require massive amounts of regeneration, he added.

On the other hand, the integration of human performance with AI also helps navigate portrait rights issues, said renowned film producer Ren Ning, highlighting the need for better industry-level digital asset protection.

The current tools cannot completely avoid infringing portrait rights, which presents an important challenge for filmmakers working with AI all over the world. "Once we get all these industry-level setups done, we can let AI empower us better and make the path for industry development smoother."

On June 19, an academic report was released on the AI Backlot initiative, written by the observation team from the School of Theater, Film and Television at the Communication University of China.

The report concluded that while AI can assist, humans still remain the ultimate creators. AI can't replace people's judgment on camera work, storytelling and audience experience. When AI struggles with handling long takes, a director's value shows in how they break down, reorganize and use editing to create a movie scene.

Meanwhile, the report found that prompts are playing a vital part. They don't replace the script, but transform characters, situations and emotions into audiovisual conditions that the computer can understand. "In the AI era, precise, expressive text is still the key to human-machine collaboration," the report said.

Yi Zhongtian, a renowned academic and adviser to the AI Backlot initiative, echoed that while AI has lowered the technological barrier to visual expression, in the era of AI, judgment, imagination and creativity — three abilities exclusively human — will be more crucial than ever.

The Decisive Moment, a film that combines live-action footage with computer-generated imagery and used AI technology in the postproduction phase, premiered at the closing ceremony of the SIFF on June 20.

Huang Jianxin, a veteran director, served as the producer of the movie. He is optimistic about the emergence of purely AI-generated films.

In the future, he said, four major categories could probably be developed in the movie industry: live-action features, animations, documentaries, and AI movies. "From a creative standpoint, AI has advantages in visualizing legendary and supernatural subjects that are hard to match with live-action filming," he said.

He said that the whole industry will be reconstructed in this new scenario. Each type of movie will develop its distinctive aesthetic system, and AI creation will develop an exclusive vocabulary, which will largely expand the artistic boundaries of movies.