Why PXO created a real-life ‘Holodeck’ for ‘Star Trek: Discovery’

The set itself is also adaptable, allowing for the inclusion of real-world props and set trimmings. So when the performers arrive on the stage, they are seeing highly polished visions of alien worlds and futuristic locales to immerse themselves in. While operating it isn’t quite as simple as hitting a button and ordering up an interactive mystery in Victorian England, or creating virtual water so you can kayak down an alien river, PXO’s virtual production stage and “AR Wall” may be the closest thing around to a real-life version of the original Holodeck. 

“Combined together, these components offer the filmmaker the ability to execute scenes that would otherwise require a film production crew to travel to distant locations, and some cases, into outer space.”
 

To create each new environment for the AR wall, the process begins in pre-production, with PXO’s Virtual Art Department (VAD) creating prototypes for each environment. To quickly create a rough representation for blocking, the team sources elements from the Epic Games Marketplace, adapts them as needed, and adds them to the scene. 

From there, PXO then breaks the design down to distribute it to a larger team of artists, all collaborating to create the build for the final environment within Unreal Engine. After that, the production team—including the director of photography—can make any changes they want. Then roughly a week before the shoot, artists apply the final touches. There may be a little post-shoot touchup work (most often around the seam between the ceiling and wall), but otherwise the work is already done.

Along with the obvious advantages, PXO discovered several hidden benefits of using Unreal Engine for virtual production. PXO was able to totally abandon its offline rendering approach to cleanup work, one of the most thankless—but necessary—parts of any VFX artist’s job. That also created a domino effect, freeing up other groups across the board.

“It's great being able to focus the team's attention on the shots they feel excited to work on rather than some of the more mundane work,” said Rahnama. "That is one of the many creative benefits for artists working in virtual production."

The lighting team especially saw a major shift. During the first season of Discovery it could take up to 14 hours for a lighting bake to finish. Now, using Unreal Engine’s GPU Lightmass Baker, high-quality bakes can be done in as little as 30 minutes.

One of the biggest, most important changes, however, is somewhat intangible. In previous seasons, each location would be divided into multiple tiles, and individual tiles would then be assigned to an artist. Once the tiles were complete, they were reviewed independently and the complete environment was only assembled for the final batch of reviews. During Season 4, PXO’s artists could all work in the same scene simultaneously. That not only led to a more cohesive and consistent environment, it also enabled artists to collaborate and inspire one another. Plus, as an added bonus, when artists know people will see their work on a daily basis, it tends to significantly improve organization.