March 18, 2020
Virtual production on the battlegrounds of “Game of Thrones”
The Third Floor is one of the entertainment industry’s most remarkable visualization companies. It has won a multitude of awards including five Emmys, and is a major contributor to the visual storytelling that audiences see today—from Marvel Studios’ films like Avengers: Endgame to streaming Star Wars series such as The Mandalorian on Disney+.
Since its inception in 2004, The Third Floor has helped content makers in film, television, video games, and location-based entertainment to design and realize compelling projects. Its teams are experts at using the virtual world to plan and problem solve ahead of shooting, leading the future wave of virtual production.
The Third Floor’s work on the mega-popular TV series Game of Thrones helped to create some of the most memorable scenes on the show. In turn, constantly striving to push the boundaries of what’s possible has had a transformative impact on the studio’s pipeline. “The experience we had on Game of Thrones gave us the confidence to adopt Unreal Engine fully across the company,” says Kaya Jabar, Virtual Production Supervisor.
Building virtual sets for huge-scale action
The Third Floor became involved in the production of Game of Thrones back in the early days of the series, previsualizing select scenes that were going to be particularly complicated. With multiple departments, VFX vendors, and units working on the show, the studio’s mockups helped define and communicate the showmakers’ creative and technical vision.With the bar for epic visuals soaring and audience expectations rising season over season, The Third Floor’s role as a hub for design and planning became increasingly important. The studio needed a way to keep improving on the benefits of visualization—it needed new tools and new approaches.
One of these tools was a virtual scouting toolset, built by The Third Floor in Unreal Engine, that enabled the show’s team to explore and stage dramatic Season 8 scenes in increasingly ambitious environments. This became a vital asset to plan out shots for sets that were still under construction or were in various phases of design. The studio deployed multiple artists to Belfast early on to work with the art department and build virtual versions of sets in Unreal Engine for scouting.
Empowering production to experiment with real-time tools
Whether evaluating a location or envisioning a yet-to-be-created set, virtual scouting enables more time for experimentation, greater discussion, and the ability to better refine the shot list. Environments can be traversed in a matter of seconds from the comfort of the art department. Testing different lenses with real-world camera settings, a variety of lighting setups, and various character animation paths allows for more creative iteration.
Compared to pre-rendered frames, a real-time workflow lends itself to flexibility, quick iteration, and constant feedback that increases creativity and promotes communication between departments within the production. "As an extension of real-time, virtual reality offers complete immersion in the environment, leading to not only an increase in artistic considerations but also aiding with technical decisions," says Adam Kiriloff, Senior Real-Time Technical Artist at The Third Floor. "For example, how many horses can fit side by side in the main street of King’s Landing? Is a crane really needed for that over-the-wall shot and how do you go about framing with a massive dragon in the room?"
The virtual scouting toolset developed in house by The Third Floor enables shots to be planned in VR. The idea behind the tool was to provide a way to view previs environments from a more immersive perspective and accelerate the process by planning camera shots and testing lenses virtually. Using either a tablet or wearing a headset, users can view the environment or be fully immersed within it.
A key feature is the “virtual lens”—a virtual screen attached to the controller in VR that users can hold up to plan shots within the virtual environment. The virtual lens mimics real-life camera and film-back settings. Lens configurations can be set in advance and lens swapping in VR is as easy as scrolling through a selection list. 3D annotation makes it possible to take notes and do group reviews. Measurements and distances can be calculated using a variety of laser and point-to-point measuring tools.
The Third Floor’s virtual scouting solution came into play on Game of Thrones in Season 8 as a way to plan camera shots and decide on character movements in sets like King’s Landing that were just starting to be constructed in the production backlot. Other sets that already existed, like the Throne Room, were virtually scouted as well in order to visualize the set in several stages of destruction and in different lighting scenarios.
Working within the art department, artists from The Third Floor built digital 3D assets and ingested them into Unreal Engine, creating environments including Winterfell, The Red Keep, and Castle Black. "We created environment materials in Substance Designer, painted bespoke props in Substance Painter, and developed particle effects and set up lighting and atmospheric effects like fog and ash in Unreal Engine," says Kiriloff. "In some instances, we used photogrammetry to capture key props, such as the Iron Throne itself."
The environments were faithfully recreated from a variety of sources and accurately reflected real-world scale. Once a version of the set was in Unreal Engine, it could be scouted by everyone from production to visual effects to the art department in an HTC Vive head-mounted display using The Third Floor’s virtual scouting toolset. This approach was key for Season 8, Episode 6—the series finale—that saw the director of photography working with The Third Floor team to virtually scout action and camera coverage to produce rough blockings.
One such sequence in this episode sees Jon Snow making his way up the stairs of the crumbling Red Keep to meet Queen Daenerys as she surveys the ruins of King's Landing and addresses her armies. This sequence began life as an animatic created by Director of Photography Jonathan Freeman as he worked with The Third Floor’s VR set scouting and virtual camera team.
Once the animatics were approved, the process moved into previs. The Third Floor’s artists took files directly from the virtual camera team and, using these as a guide, began to add animation and camera moves to the shots.
For the pivotal Throne Room scene, the episode cinematographer scouted a virtual version of the Throne Room created by The Third Floor. Working in the environment in real time, it was possible to quickly find the best possibilities for shots and angles that fit for the story, taking still-frame photoboards on the fly to develop a refined shot list. In a scene featuring a 747-sized CG character within a real set, virtual scouting provided an effective way to evaluate the area the dragon would take up and what frame-ups worked best to capture the action.
The interactivity inherent in real-time technology provides the power to visualize and evaluate while being accurate to production’s evolving designs, helping to inform them. With wait times eliminated, it was not an issue to try out an idea or see what the scene might look like with different lighting options or even at a different time of day. This put the project into a more flexible space, with more opportunity to explore new creative avenues.
Empowering creatives with new virtual toolsets
Clients love the immediacy of real-time rendering, but also want uncompromised visual quality. This is one of the key reasons The Third Floor adopted Unreal Engine in its real-time workflow. “If we're going to be catering to cinematographers, directors, and designers who demand the highest fidelity, the best lighting, the best overall user interface, we need a workflow with Unreal Engine,” says Chris Edwards, CEO of The Third Floor.In the end, it comes down to that ability to facilitate greater creativity. “For me, Unreal Engine is the ultimate holy grail that helps bring together everything we are trying to do, allowing us to provide directors, designers, and other visionaries with much more capability than they could have ever imagined,” says Edwards. “They now have so much more freedom to create whatever they want.”
The Third Floor foresees many more creators in entertainment catching on to these benefits in the future, even for smaller-scale productions. “You know, a lot of people think that this is a big, expensive thing because our work includes Hollywood blockbuster movies,” says Edwards. “But really, there’s so much potential for many more films and projects, and I think Unreal Engine is going to unlock mainstream access to this type of toolset and capability.”
Want to test out and explore your own creative ideas? Download Unreal Engine for free today!