FOX Sports kicks off the NFL season with a groundbreaking multicam virtual production studio

The start of a new era

For Executive Vice President and Creative Director at FOX Sports, Gary Hartley, the move to the new facility is just another example of FOX’s innate ability to combine storytelling with unique visuals. “When the sports division started, it was run by a guy named David Hill and his approach was innovative,” Hartley explains. “He would always question things. He’d ask ‘why don’t I know what the score is?’ and get broadcast graphics to help add it to the screen. Back then, our mission statement was ‘same game, new attitude.’ That has been the underlying ethos at FOX Sports for almost 30 years: To not only show the game, but push the envelope of what a sports broadcast can be.”

This ethos was evident in 2019, when FOX Sports became one of the first broadcasters to make significant inroads into game graphics after seeing Unreal Engine technology demonstrated at NAB. The resulting virtual NASCAR set in Charlotte, NC gave the team the perfect opportunity to experiment with real-time technology. The only problem? The studio was constructed with green screens rather than LEDs. That meant issues with green spill and no way for talent to see or interact with the visuals on set. 

“Before long, we noticed there had been a lot of noise made around LEDs and XR,” remembers Senior Vice President of Graphic Technology and Integration at FOX Sports, Zac Fields. “We knew people were using LEDs in the film and episodic world, but no one was doing it live yet. We saw a couple of early demos and knew we had to try this new technology out. By using LEDs, our team could easily bring talent to any environment and get photoreal lighting effects on set, without having to worry about keying a green screen or pre-rendering complex elements like player graphics ahead of time.”
 

“The LED wall at the new studio is 10,000 pixels across without even counting the floor. It's just a massive thing to output to 4K,” says Alex Seflinger, Lead Technical Artist on the project. “It was already a complex setup, but we wanted to do even more. We wanted to make the volume work with multicam.”

To do this, the team used GhostFrame from MegaPixel: a new technology that can combine four different 60 Hz video streams to a 240 Hz refresh rate LED wall. Once the engineering team sets each frame, a director can then see four separate camera feeds with their own unique perspective, all of which appear in real time. 

“When we first thought of using multicam on a virtual production, we thought it would be impossible,” says Fields. “Luckily, we had GhostFrame technology. It was the secret sauce to making the live broadcast happen.”

In order to minimize latency and the multiple points of failure involved in the large setup, the FOX Sports team built each Unreal Engine machine to be identical, with the exact same internals and graphics cards. The only change was the part of the volume each engine was feeding, depending on whether it was AR elements or the LED walls and floors. 

“A big thing I’ve noticed with Unreal Engine is that it is a very stable product day in and day out,” Moore adds. “We could build rich, layered scenes that had virtual graphics everywhere from the floor to the sky above the talent. That meant we could tell better, more dynamic stories, all while knowing we could trust Unreal Engine to render it all without crashing live on air.” 

Less than a month after the show’s debut, the stage has been such a success that the team has plans to use it for other sports like baseball and basketball, and there are already plans in place to experiment with more interactive AR. “We see ourselves as storytellers and Unreal Engine is giving us the tools to tell our stories in brand new ways,” Fields concludes. “Five years ago we saw the potential of real-time technology with our NASCAR stage. Now, with our new LED volume, we can do just about anything.”