What is VP used for?
Virtual production (VP) is an evolution of established film and television production methods with as many advantages as potential obstacles. With VP Scenes, or Volumes, all over the world, filmmakers experienced with its tools want to share both the pros and cons to keep virtual production on track.
“VP means a lot of things to a lot of people. It’s a very broad umbrella,” says Callum MacMillan, Virtual Production Supervisor, Dimension Studio. “At its core is a real-time game engine and the representation of 3D worlds on a giant LED wall to create the illusion of a real environment.
In this sense, VP is an extension of classic cinematic techniques, such as front and rear projection or green screen, in which actors are filmed against artificial backgrounds for greater profitability and control.
“It’s the next evolution in visual effects,” says John Rakish, president of Location Managers Guild International. “What’s radically different with virtual production is the introduction of an interactive light source that allows the cinematographer to capture the final images behind closed doors.”
The Mandalorian, for example, featured a central character with a reflective jumpsuit that could be photographed with light cast from the surrounding LED panels.
VP should also be understood as a set of tools and processes that extend beyond the performing environment. Applications range from remote previewing and scouting to character creation based on performance capture. Assets such as models, characters, and 3D environments must be fully camera-ready before production begins.
“There’s no guesswork to ‘fix it in post’ when you can crop a shot on set based on what you see through the lens,” says production manager AJ Sciutto. virtual at Magnopus, a virtual production company based in LA and St. Albans. “That production value alone, if done well, is a key motivating factor.”
“The pandemic has accelerated demand,” says Sciutto. “There was a lot of motivation to get back to work safely without sending hundreds of crews on site.”
The practical advantages of virtual
Even before Covid, VP was heading into the mainstream. Indeed, the predictability of the virtual environment leads to cost savings in the end.
“All of the variables of on-location filming, such as weather conditions or the actual location of artisanal services, can be controlled,” says Sciutto.
“Just not unloading and repacking the kit every day saves time,” says Max Pavlov, CEO and co-founder of LEDunit, Vice President of Riga, Latvia. “You can shoot on golden hour all day if you want. There’s no need to stop and wait for the perfect light or worry about paying overtime.
According to Deloitte, location-based reshoots, which can account for 20% (or more) of the final production cost of big-budget movies, can be eliminated with careful advance planning.
VFX costs on a big-budget sci-fi or fantasy film can run up to 20% of the film’s total budget, but filming against an LED wall dramatically reduces compositing and rotoscoping overhead.
“With game engines, we can put all of the individual aspects of a linear pipe into a single environment to coexist at the same time,” says Ed Thomas, Dimension’s vice president manager. “Potentially you have production design, lighting, layout modeling and animation all happening at the same time in one environment.”
VP scenes have found a natural home for filmmakers wanting to bring fantasy story worlds to the screen, but the technology is equally applicable “from spaceship to pack shot”, says the cinematographer and cameraman , Richard Dunton.
What content works best?
Virtual production opens up the possibility of filming in hard-to-reach areas, or simply downtown a city where permits would be prohibitively expensive. Small VFX teams can use photogrammetry and Lidar scans to reconstruct these locations in a studio.
Scenes of conversations in moving cars, walking and talking, “things you’d spend a lot of money to shoot on location can be shot with virtually the same quality,” says Sciutto.
Another benefit is the edge it gives actors who can tailor their performance to places they can see rather than unseen creatures or environments that require greater leaps of imagination.
“The actors’ ability to play in true-to-life environments is the real goal of virtual production,” says Philipp Klausing, MD at Dark Ways, who just produced the Netflix period drama 1899 on a virtual stage in Berlin. . “We record the performance, not the background.”
VP is not a silver bullet
Despite all its advantages, VP does not make creative or commercial sense for every production.
When director Denis Villeneuve invited The Mandalorian cinematographer Grieg Fraser to photograph Dune, it wasn’t to tap into Fraser’s virtually unique expertise, but the gritty realism he brought to movies like Zero Dark Thirty.
Both Fraser and Villeneuve wanted audiences to feel the heat and sand of a desert while filming in the Middle East.
“VP is not a silver bullet,” says Sciutto. “There are a lot of people trying to force things into a VP workflow. There’s absolutely a tactility to places that you don’t get inside a volume.
Shooting close-up LED screens can lead to problematic visual artifacts, which is why shots in a volume tend to have a blurred background with shallow depth of field.
Framestore episode manager James Whitlam advises that a dialogue-heavy drama to be filmed in tight close-ups would be too expensive on a virtual stage, but that “one volume is perfect for recurring scenes over multiple seasons where you can achieve economies of scale. ”
The Game of Thrones sequel House of the Dragon, for example, was shot entirely on a Warner Bros volume in Leavesden.
Turning in a volume isn’t all or nothing either. Stages made up of modular and portable LED panels can be built to specifications.
“A scene set on a train, for example, may only need enough panels to cover the windows to give the realistic reflections of your interior,” says Chris Hunt, chief financial officer at the specialist. LEDs, Brompton Technology.
The Mandalorian and 1899 used regular soundstages alongside a volume. This can be to speed up schedules (shoot different scenes simultaneously), or for aesthetic reasons. “Shooting an entire show in one volume can feel too sterile,” says Sciutto.
The physical constructs and props on the volume stage must merge convincingly with the CG backdrop, which puts additional pressure on the production design.
“The entire LED image in the camera should pass seamlessly between the floor, ceiling and wall of the volume,” he says. “Most LEDs are at 90 degree angles to the stage and require paint cleanup after the fact. Placing physical sets, such as columns, that repeat seamlessly in the wall works great because the camera and your eye can fixate on the content that is lit in the real world.
Mix physical setting and virtual world
Volume stages aren’t necessarily cheap either. Depending on the scope of the project, creating VFX may require additional manpower. The highest cost is for LED panels, originally built for DooH and currently expensive rental items.
“Everything is bespoke and there is no assembly line methodology to assemble a volume stage,” says Scuitto.
For all the amazing photorealistic quality of game engines like Unreal, the software is still not capable of rendering some of the highest quality infographics. Elements such as human background characters, rainforest foliage, and waterfall simulations are considered distracting.
“There are many interwoven parts of VP that can create cascading problems if one element is misjudged or overlooked,” says Grieg Fraser, director of virtual and adaptive production at the Entertainment Technology Center in Los Angeles.
“For example, the idea that LED screens themselves can illuminate actors and scenery is largely true, but it masks a maze of complications and decisions that the cinematographer and director must wrestle with.”
Technology aside, the biggest impact of virtual production is the disruption it means to conventional filming. New production roles and lines of decision-making must be agreed upon and understood.
The Virtual Art Department (VAD), for example, headed by the VAD Supervisor, will include those responsible for creating lighting and assets in the virtual world. Their work must interface with a gaffer and the production designer (who works on both real and virtual sets). The ultimate responsibility for the composition of the picture rests with the DP and the director.
“There is a well-established hierarchy on a normal set. You plug in virtual production and that changes,” says James Whitlam, MD, Episodic, Framestore. “In 1899 we got into a stream where we weren’t stepping on each other’s toes, but we would have had to do ten such works to really know if this is a working model.”
Various bodies are working to agree on job descriptions and pay rates. A new lexicon for creating in virtual production is being developed by organizations such as Netflix, SMPTE, Disguise and VES.
An initiative by SMPTE aims to create a library of digital assets that producers can quickly license and download. This is becoming more and more urgent in order to reduce the pre-production time needed to build virtual worlds, especially for producers who have to convince investors to finance the production.
“I’m looking for an independent library of geotagged data and a unified technology standard that will allow producers working anywhere to pick assets and download them, with a few tweaks, ready to play in any volume,” says Klausing.
Beyond that, there are plenty of educational initiatives to get experienced filmmakers and graduates familiar with VP scenes.
“Just like when the industry moved from shooting movies to digital, some directors and cinematographers need some convincing (about VP),” says Pavlov. “But when they try it, they understand how easy it is.”
Visual effects producer turned director Paul Franklin is a convert. “Once in a digital world, your creativity is infinitely malleable. For me, virtual production is a way of taking what I would have done as a VFX artist and putting it live on set.