Pixomondo gives viewers a kick with new virtual production-based soccer commercial

Back in the golden age of Hollywood, if a director needed a crowd for a scene, they’d have to bring in real actors. Ben Hur’s famous chariot race saw 8,000 screaming extras fill the stadium, for example. By the time Gladiator was released, it was possible to use CG to populate the Colosseum. The former approach is an expensive, logistical nightmare; the latter can involve hours of laborious digital artist work. 

When you’re shooting something like a commercial, budgets are lower than for Ben Hur. Renting a stadium and bringing in hundreds or thousands of extras is off the table. Spending hours in post-production is an option, but not a particularly encouraging one.

Fortunately, there’s now a way to create CG crowds that are indistinguishable from the real thing at a fraction of the time and expense. 

On a recent commercial for the Caledon Football Club, Pixomondo (PXO), alter ego, William F. White International (WFW), and Virtual Production Academy (VPA) showed how, cutting costs—not quality—with an LED volume and virtual production workflows.
 

“As far as I am aware, slow motion has never been achieved on an LED stage before this commercial,” says Whiteson. “When running higher frame rates, like 48fps on LED walls, the likelihood of seeing scanlines in-camera becomes more prevalent. However, this is also contingent on your LED panels, cameras, and the movement of these cameras. To achieve the best results, we tested three different cameras at various different frame rates while the wall was running at either 24fps or 48fps.”

In the end, the team decided to use the Alexa Mini because it can do 200fps and produces the least amount of flicker. “No other camera other than Phantom could do that for us without cropping in on the sensor,” says Whiteson. “Even at 200fps, we still needed to slow it down to 800fps.”

In addition to the flicker, the team also discovered that the crowd simulation would need to run slower than the original 24fps. Once they shot their first test, it became clear there was not enough information in the crowds movement to withstand the 200fps camera speed, so they prebuilt different speed variations of CG crowd. At any given moment, they could instantly activate one of the animations, depending on the frame rate they were shooting at.

“That's a huge benefit of using Unreal Engine—to load those different assets at the push of a button,” says Whiteson.
 

Digital 3D crowds for film and TV

As a compositor, Whiteson is often asked to duplicate or generate 3D crowds to fill a stadium—a task that he says is both time-consuming and challenging. “It’s always tedious work,” says Whiteson. “Tracking, keying, or rotoscoping, cloning shot elements, blending in atmosphere—it’s something we are asked to create maybe four to five times a year.”
 
The team at alter ego wanted to provide a cost-efficient solution to clients that would give them the freedom and creativity to tell their story—without huge post implications that tend to follow crowds.

That’s where PXO came in. Earlier this year, the award-winning international visual effects and virtual production company completed over 700 VFX shots for the HBO series Winning Time: The Rise of The Lakers Dynasty. 

As you can imagine, 90% of the work consisted of CG stadiums and crowd simulations. The PXO team realized there was a need for a faster and more efficient way of doing 3D crowds. In the film and TV industry, ‘faster and more efficient’ often means using game engine technology, so the team developed a flexible crowd tool in Unreal Engine that could not only perform in real time, but be used on LED volumes.

When the opportunity to work on the Caledon Football Club commercial came up, alter ego and PXO knew they had to put this new tool to the test and really push it to its limits.

“PXO and alter ego have collaborated on several commercials,” says Pixomondo Chief Innovation and Chief Creative Officer Mahmoud Rahnama. “After PXO wrapped the first season of HBO’s Winning Time, I told David about the challenges we had creating the VFX crowd sequences using traditional methods. He told me about his challenges attempting to shoot high frame rate shots on an LED wall. That’s when we decided to score two goals with one ball!” 
 

The PXO’s crowd system works by leveraging efficient static mesh instancing in the engine. All animation was done in Vertex Shader using bone animation textures. The system consists of two parts—a Houdini toolset for model and animation processing along with bone animation texture baking, and an Unreal Engine plugin, written in C++, to generate and control the crowd instances.

For those moments when the crowd needs to move as one, the system supports triggering different crowd emotions. There are over one hundred different animation clips that can be used on any crowd person, which can be activated to affect the whole crowd or just one section of it. The system also supports smooth blending between animation states.

“The ability to control the crowd to perform exactly as needed on cue is huge,” says Whiteson. “We pre-animated seven different crowd performances to be used depending on the action of the play on the field. If we wanted the crowd cheering, we just loaded up that specific asset. Likewise, if we wanted them to be sitting and not too excited, that was also a pre-animated simulation. Much easier than trying to wrangle real life people to perform in unison and on cue.”
 

Prior to LED walls, a project like the commercial for Caledon Football Club would have required alter ego to bring between 50 to 100 people into a real-life stadium, move them around from section to section, and then put it all together in post.

Or they would have to generate the crowd in CG, track the footage, rotoscope any foreground assets, and try to integrate it all together by matching lighting and atmosphere. “If creating a CG crowd needs to happen either way, why not spend the time creating it before the shoot and use Unreal Engine to animate in real time on the LED wall?” says Whiteson.

This way, the team is free to move its camera however it chooses because everything captured in camera is final pixels—what’s more, the blending between the real world and the environment on the wall is something that can not be matched in post. 

“The stadium lights in the Unreal environment reacted in the lens just as real stadium lights would,” says Whiteson. “The bokeh and light wrap around our athletes that we captured in camera was as true to life as if we were in a real stadium. Again, this is something that’s time consuming and difficult to replicate in post production.”

Chadwick points out that working with a LED screen also saves huge amounts of time compared to traditional techniques. “One of the main advantages of shooting a project like this on a LED volume is the speed at which we were able to execute,” he says. “You can imagine that if you were shooting this in a real stadium, aside from being expensive, it would take very long to reset for each take. By leveraging the real-time render capabilities of Unreal, we were able to reset with just the click of a button.”
 

LED stages make impossible shoots possible 

Whiteson believes agencies and production companies have good reason to be excited about virtual production techniques and LED volumes. 

He explains that they are currently working on a project that would be impossible to complete without Unreal Engine and an LED wall. The story needs to take place in winter but the team are shooting in September. That puts four options on the table.

The first would be to fly to a far-off location where it is currently winter—incredibly expensive. 

Alternatively, they could shoot at a local ski resort at the end of the summer and then replace the entire background in post. This option is very costly and time consuming in post production. 

The third option would be to shoot in a studio on a blue screen. “With this option we are dealing with studio lighting and 3D post environments, and the integration never really looks perfect,” says Whiteson. “Additionally, the delivery schedule does not allow for much post time.”

The final option was to create the environment before the shoot in Unreal Engine and use an LED stage. “For us, the last option will produce the best results,” says Whiteson. “The LED wall is so powerful in its brightness that it really does do a lot of the lighting for you. 

“Plus, the integration between the talent and the background would look far superior to any composite. We can also use atmosphere and snow falling on set which could never be done in a blue screen environment. Have you ever tried to key falling snow? It’s a nightmare. For the budget and timeline, the LED wall was really the smartest option.”