May 1, 2019

Virtual production: Stargate Studios creates final pixels on set

Film & Television

Stargate Studios

ThruView

VFX

Virtual Production

By Brian Pohl

After three decades in the business, Stargate Studios (The Walking Dead, CSI: Crime Scene Investigation, Ray Donovan, The Orville) knows a thing or two about high-tech film and television production. Offering both visual effects and digital production services, the company now has nine studios in seven countries, and a rich legacy of experience to draw on. But, as CEO and Founder Sam Nicholson, A.S.C. tells us, the company is never complacent. “We're always trying to reinvent ourselves,” says the distinguished cinematographer and VFX supervisor/producer. “We're trying to think ahead, and be relevant—five years out.”

So what does that mean in today’s fast-moving industry? For Stargate, for those visual effects that can be done in real time, it’s about bringing them right on set, and making them part of the filmmaking process.

To achieve this, they’ve created ThruView, a set of tools that enables them to shoot in real time, while actually seeing CG elements—complete with reflections and lighting—integrated in the camera, eliminating the need for green-screen setups.

The company’s President, Al Lopez, explains: “At the heart of it, it’s a system that uses photographic plates,” he says. “We are shooting multiple 8K cameras. You can combine it with CG elements and it's going to be a hybrid: everything from full CG, if you want synthetic, all the way to fully photographic.”

The combined output is piped into Unreal Engine at high resolution, playing back at up to 60 frames per second. “That is a lot of data that has to get through and be processed in real time,” says Lopez.

Nicholson elaborates: “What the Unreal Engine allows us to do is put everything in one basket and shake it up, and when it comes out, it looks real.” he says. “3D, 2D, real-time color, tracking, off-axis rendering, all these things [...] happen in real time. And it gives us enough hooks into it that with tablets, we can be adjusting color on the fly, we can be adjusting the lighting on the fly, we can be doing ray tracing on the fly. The Unreal Engine is the perfect software complement to all these converging technologies.”

Clients are suitably impressed. “We're showing people something they have never seen before,” says Nicholson. “You generally don't see very many things for the first time. It's real magic when you see it.”

Bringing VFX on set helps the crew and cast feel more connected to the end result, as Producer Bryan Binder explains. “Let's say you're doing a show that takes place primarily on a bus, and instead of sitting there for day after day after day staring at green screen through the windows, and the entire crew and all the cast being so disconnected from what's going on and what this thing is going to look like at the end of the day, we’re actually able to bring that experience to everybody live on set,” he says.

But the benefits are also financial, with a show shot in even a single environment or just a few environments potentially generating over 500 green-screen shots. “At that point, now we start talking about the financial realities of shooting on green and the financial realities of what ThruView can offer a production,” say Binder.

Nicholson is keen to point out that, while real-time on-set effects are not new, producing final-quality pixels from them is. “For many years we've pursued real-time green-screen shooting; there's a four- or five-frame lag on there and you're basically doing a previsualization on set, but it's not a finished product,” he says. “Now what we're doing is going for finished product on set in the lens—done.”

Because of the company’s post-production expertise, Stargate has been able to reassure nervous clients that they can always fall back on traditional methods at no extra cost if the real-time results are not satisfactory. “So far, nothing’s come back,” says Lopez.

According to Nicholson, Unreal Engine has played a pivotal role in ThruView’s success. “I don't think any of this would be possible without the Unreal Engine,” he says. “There's a universal creative language developing; the Unreal Engine is very much part of that. It’s the converging of all these technologies: individually they're all getting faster and better, but what the Unreal Engine allows us to do is put all of them together in one place at one time in a dependable fashion—on set.”

Want to find out what magic you can create in real time? Download Unreal Engine today.