Netflix is pioneering a new technique that seeks to revolutionize the age-old process of compositing in filmmaking. Using a combination of machine learning and a striking magenta lighting setup, the process, dubbed "Magenta Green Screen", simplifies the cumbersome task of replacing a brightly colored background (blue or green) with the desired scene, a method known as chroma keying.
A recently published paper reveals that Netflix's Magenta Green Screen achieves impressive results by creating a lighting sandwich - a mix of red and blue lights in front of the actors and bright green lights behind them. While this provides a stark contrast in colors, it simplifies the process of separating the foreground from the background.
However, this innovative method brings its own set of challenges. While compositing becomes easier, the process of restoring the green channel to the magenta-lit subjects is complex. To overcome this, Netflix's team trained a machine learning model on rehearsal scenes lit normally. The AI model, using a convolutional neural network, compares full-spectrum image patches to the magenta-lit ones and develops a systematic approach to restore the missing green channel intelligently.
To address the issue of unnatural on-set lighting, the team proposes "time-multiplexing" the lighting, where the magenta and green lights switch on and off multiple times per second. If done rapidly enough (144 times per second), the lighting appears nearly constant.
While still experimental, this new approach offers a fresh, high-tech solution to an old problem in media production, potentially simplifying compositing while maintaining high quality results. It's a testament to the power of AI in shaping the future of film and TV production.
