LED VIRTUAL PRODUCTION

Virtual Production (VP) is the ‘umbrella term’ for describing 6-8 variations of creating composited visual effects in real-time, by combining camera-tracked computer graphic imagery (CGI) with real camera footage.

In the TV/cinematic world they use the term ‘real-time VFX’ (real-time visual effects) or ICVFX (in-camera visual effects) because the composited visual effect shot happens in front of the camera in real-time, rather than in a computer in non-real-time.

In simple terms then, virtual production is the blending of a CGI scene or object with a physical camera feed, using camera tracking as the glue, to create real-time VFX content in the camera.

Virtual Production has been in existence since the mid-1990s, and up until about 6 years ago it was used exclusively for broadcast productions, typically sports, news, and weather. In this version the graphics engine creating the virtual studio was basic, certainly not photo-realistic, and the camera tracking used to blend the virtual studio to the camera movement, was neither sufficiently accurate nor did it enable full unrestricted camera movement. It also required a green screen studio to enable the talent to be separated from the physical studio, and for the virtual CGI studio to replace the green.

Around 2016 people began experimenting with Epic Games’s Unreal Engine to create far more photo-realistic virtual studios (VS) or augmented reality (AR) content for cinematic productions using blue/green screen studios. At that stage the use case was to avoid making VFX errors on set that were costly to fix in post-production, or even worse, they required a re-shoot. The system used proxy quality CGI assets, and it worked brilliantly in enabling productions to see in a monitor, and then fix on-set, any eyeline or positional VFX errors. But this system still required the lengthy and costly post-production compositing stage after shooting. The goal then was to move to real-time visual effects using full quality CGI assets.

Although not the first production to use LED screens or ‘LED volumes’ for virtual production, Disney’s “The Mandalorian” released in November 2019 changed cinematic features/episodic TV production forever. For the first time, two key technological innovations  came together – full quality photo-realistic CGI scenes used in real-time, being displayed on an LED screen or ‘volume’. For the first time this enabled what was shot on set to be the complete VFX shot captured ‘in-camera’, which was referred to as ‘final pixel’. It wasn’t quite that simple in the end, but the concept of shooting visual effects shots in real-time was firmly established, as was using LED screens to display the virtual scene instead of using blue/green screens and adding the virtual scene in post-production.

Today the technology has developed further. The LED tiles are more ‘cinematic’ in their colour and light properties, the Unreal Engine virtual scenes are even more photo-realistic thanks to developments in graphic processing cards (GPU), and the camera tracking – the glue that blends the virtual CGI world to the real world that the camera sees – has become more accurate and is now able to accommodate virtually any camera or lens movement.

Today, this same virtual production technology, along with all of the benefits it provides, is available to the corporate sector. But not every manufacturer or systems integrator can offer a complete solution, in fact very few can. Bendac is one of the few companies who have the background and experience to offer complete LED VP studios for the corporate sector.