AI Automation in Filmmaking: Roll camera – AI – Action!

PICTURE COURTESY: Zohar Lazar

Remember Jarvis, Iron Man’s AI-backed companion with British accent and an obedient “at-your-service sir” attitude? How astounding it is to witness a tech-world where imaginary characters can become a reality with the help of AI. Facebook founder, Mark Zuckerberg has already started to work on the subject by inventing his personal home AI assistant called ‘Jarvis’ to create virtual reality visualizations.

It all began with Expressionism 

Experts are working persistently to experience pivotal Artificial Intelligence (AI) age not from a decade but from far earlier. Film industry is one such arena utilizing AI in big-small ways to add special effects and gather eyeballs. Surprisingly, the use of AI in movies dates back to the time where before the videos even had audio. Shocking, right? In 1927, German Expressionism film ‘Metropolis’ introduced a robotic character which became a benchmark and allowed experts to forecast the success of AI in film making and advertising in the years to come. Today, the technology providers have upped the ante and companies are training their AI systems and bots to make a complete video. Use of sophisticated technology has allowed the film makers to evade time-consuming activities while saving significant production costs. This is evidenced by many movies which have become benign examples of this smart technology.

Movie Trailer for ‘Morgan’ 

20th Century Fox collaborated with IBM and its AI system called ‘Watson’ to develop the trailer of the horror movie ‘Morgan’. The 6 minutes trailer was created by Watson in only 24 hours that could have taken weeks if produced by humans. The AI system was trained by the experts by segmenting audio, visuals and other essential elements of 100 horror movies to teach it moments for creating automated trailer. Each shot was labelled with an emotion from 24 various emotions like frightening, eerie, and joy to make the trailer functional. The movie was the first attempt at using the AI for trailers and it not only reduced production cost but also the time from weeks to hours.

‘Zone Out’, the automated short film

The fact  is, the movie is fully bizarre, has incoherent and hilarious dialogues, a completely muddled plotline, distorted characters, and eerie soundtrack. However, despite being a mismatch for entertainment, the video demonstrated a positive indication of AI in the entertainment sector. This 5-minute film was written, directed and edited entirely by the AI algorithm called ‘Benjamin’ in just 48 hours. The creation is a collective effort of Los Angeles based director, Oscar Sharp and Ross Goodwin, who is also a creative technologist at Google, to learn dialogue using voice-generation and face-swapping technology. The partners have worked before to work on the same subject by creating another short film- ‘Sunspring’. The movie was less of a failure than ‘Zone out’ as it used real actors and dialogues. 

The Endgame’s AI Paradigm

Shout out to all the Marvel fans! James Alexander Hendler, an AI researcher with his expert team used AI algorithms and machine learning programming to convert the very dashing Josh Brolin into the anti-hero- ‘Thanos’. The team worked extensively to train the AI-backed system to scan and record actor’s expressions and face structure to automatically map on the animated character body. This helped the actor to perform with other co-stars instead of alone in front of a blue or green screen. Meanwhile, Digital Domain’s machine learning algorithm saved lots of time by capturing it in real time to perform face mapping and swapping.

AI and robots are rapidly automating the filmmaking process to bring out impressive results and error-free edits but there is still humongous amount of work to be done. Technology is improving and there is a long way to go and develop smart AI algorithms to speed up the entertainment sector but experts like Hendler believe that soon AI and bots will become the norm to film industry. Audiences and fans of expert CGI and animation just have to sit tight and cheer for more that’s coming their way.

Leave a Comment