Behind the scenes on Weta Digital’s real-time hair and fur short Meerkat
Mike Seymour |
December 10, 2020Innovative New Zealand-based visual effects house Weta Digital knows a thing or two about CG hair and fur. The multi-Oscar-winning company’s filmography includes such masterclasses as King Kong, The Jungle Book, and The Planet of the Apes franchise. As real-time technology becomes an increasingly important part of the filmmaking process from previs to final pixels, Weta is exploring just how far it can push movie-quality hair, fur, and feathers for linear storytelling using real-time tools. The result is a short film called Meerkat, rendered entirely in Unreal Engine.
Get the Meerkat sample project today!
Now you can see exactly how Meerkat was made; we’re making the entire scene available as a free sample project you can download today.
You’ll get the fully rigged, groomed, and animated characters—together with their environment, cameras, effects, and audio—plus the Maya files, final textures and materials, and more.
Weta has been working closely with Epic Games to help define and refine Unreal Engine’s strand-based hair and fur system since its initial release in 2019, and the team’s experience and knowledge, together with the insights it gained while making Meerkat, have helped to make the toolset production-ready for simulation and rendering in real time in Unreal Engine 4.26.
To those familiar with Weta’s body of work and the advanced, proprietary tools that have won it international acclaim, it’s no surprise that the team could produce such a stunning piece. But what makes this project really remarkable is that it was created by a small number of artists using out-of-the-box tools. Meerkat is a demonstration of what anyone can do, if they are talented enough storytellers and gifted enough digital artists.
Here, we take you behind the scenes of the making of Meerkat.
Over the last four or five years, Weta Digital has been exploring the interactive real-time immersive space and looking into content creation in Unreal Engine. While Weta is also using Unreal Engine in its virtual production research, for this short film the objective was to test just how well the team could use the latest standard Unreal Engine tools to produce linear content.
“We didn’t set out trying to make this a major production. We wanted to emphasize the workflows and processes behind the scenes and just see where we landed,” explains Keith Miller, Creative Director and VFX Supervisor for Weta’s Special Projects team.
Weta has done a lot of innovative hair and grooming work, developing some of the world’s most advanced traditional feature film hair and fur assets. In addition to exploring the standard Unreal Engine tools—rather than using the studio’s proprietary pipeline—the main difference for this project was the focus on real-time workflows. The team was keen to discover what it meant to the animators to be able to work with the characters with fur in place, and to iterate on shots with instant feedback.
The team started testing initially in 2019 using two characters, Caesar and Maurice, from The Planet of the Apes films. While these characters represent the high-water mark in digital fur, instead of using the existing complex assets, the decision was made to really test the pipeline by building new characters from scratch using publicly available software.
“We didn’t set out trying to make this a major production. We wanted to emphasize the workflows and processes behind the scenes and just see where we landed,” explains Keith Miller, Creative Director and VFX Supervisor for Weta’s Special Projects team.
Weta has done a lot of innovative hair and grooming work, developing some of the world’s most advanced traditional feature film hair and fur assets. In addition to exploring the standard Unreal Engine tools—rather than using the studio’s proprietary pipeline—the main difference for this project was the focus on real-time workflows. The team was keen to discover what it meant to the animators to be able to work with the characters with fur in place, and to iterate on shots with instant feedback.
The team started testing initially in 2019 using two characters, Caesar and Maurice, from The Planet of the Apes films. While these characters represent the high-water mark in digital fur, instead of using the existing complex assets, the decision was made to really test the pipeline by building new characters from scratch using publicly available software.
Image courtesy of Weta Digital
The characters were modeled in Autodesk Maya and their fur groom was created with Yeti, a Maya plugin from Peregrine Labs. The groom was then exported as an Alembic file and imported into Unreal Engine.
With the initial testing proving successful, the team turned to creating the story for the new short piece, which they decided would involve a meerkat they named Molly and her eagle foe. To get accurate reference material, the Special Projects team visited Wellington Zoo, with which Weta has a close working relationship, and studied real meerkats.
Once the story idea had been developed as a traditional storyboard, the team blocked it out at a simple quality level it calls “chess piece previs / motion staging.” This informed not only the animation but also the environment work. While the burrow itself was a custom model, the background and surrounding rocks were all built using Quixel Megascans, which are free for all use with Unreal Engine; these assets were active in Unreal Engine in case the environment needed adjusting or “dressing” to a particular camera angle.
At the early stage of making characters and animating them, iteration is the biggest factor. The animators at Weta were able to open a scene in both Maya and Unreal Engine, animate Molly in Maya and view the real-time rendered results in Unreal Engine. This is possible due to Unreal Engine’s Live Link feature, which provides a common interface for streaming and consuming animation data from external sources into the engine.
With the initial testing proving successful, the team turned to creating the story for the new short piece, which they decided would involve a meerkat they named Molly and her eagle foe. To get accurate reference material, the Special Projects team visited Wellington Zoo, with which Weta has a close working relationship, and studied real meerkats.
Once the story idea had been developed as a traditional storyboard, the team blocked it out at a simple quality level it calls “chess piece previs / motion staging.” This informed not only the animation but also the environment work. While the burrow itself was a custom model, the background and surrounding rocks were all built using Quixel Megascans, which are free for all use with Unreal Engine; these assets were active in Unreal Engine in case the environment needed adjusting or “dressing” to a particular camera angle.
At the early stage of making characters and animating them, iteration is the biggest factor. The animators at Weta were able to open a scene in both Maya and Unreal Engine, animate Molly in Maya and view the real-time rendered results in Unreal Engine. This is possible due to Unreal Engine’s Live Link feature, which provides a common interface for streaming and consuming animation data from external sources into the engine.
The team could view the scene from all angles in context with the lighting, materials, and animation. “You can really see where things break down, or what areas need improvement, and just iterate really fast without the overhead of a normal film pipeline,” explains Senior Digital Artist Nathan Farquhar, who modeled and groomed the creatures.
“Having the Live Link is part of what made this project possible,” adds Thelvin Cabezas, CG Supervisor. “We adjust our animation controls in Maya and see what it looks like in Unreal Engine and this gives us a really controlled approach to the way we work.”
Away from the shot-specific work, the animators developed the animation rigs they needed to get the right performance from the two creatures. The eagle’s wings were particularly complex, as they needed to correctly fold and come into the body, as well as be seen in full flight. Farquhar worked out the complex setup for the geometry-based rachis or spines as part of the eagle’s skeletal mesh, enabling the team to bind the barbs as strands and produce the beautiful feathers on the eagle.
“Having the Live Link is part of what made this project possible,” adds Thelvin Cabezas, CG Supervisor. “We adjust our animation controls in Maya and see what it looks like in Unreal Engine and this gives us a really controlled approach to the way we work.”
Away from the shot-specific work, the animators developed the animation rigs they needed to get the right performance from the two creatures. The eagle’s wings were particularly complex, as they needed to correctly fold and come into the body, as well as be seen in full flight. Farquhar worked out the complex setup for the geometry-based rachis or spines as part of the eagle’s skeletal mesh, enabling the team to bind the barbs as strands and produce the beautiful feathers on the eagle.
It was extremely complex for the animators to produce an eagle that did not have collision issues with the feathers and would run in real time. While Molly was groomed with 400,000 strands, the eagle ended up with over 3.7M strands. In the past, the artists might have had to resort to using cards to address a bird’s wings in a game engine, but with the new Hair Rendering and Simulation system in Unreal Engine 4.26, they were able to produce feature film-level, fully accurate feathered wings.
The turning point for many of the team was a turntable walk cycle that Animation Supervisor Ludovic Chailloleau created for Molly. While they had gone into the project with confidence, the project was always going to be pushing the new hair and fur technology. But Chailloleau’s subtle animation, paired with shader work done by Cabezas, revealed to the team just how impressive the results were going to be.
“Thelvin had done some amazing work on the shading and when we just saw Molly walking on the spot, it was really convincing,” Farquhar recalls. “That was the point I was convinced it would work.”
The shaders in Unreal Engine turned out to be very similar to those the team was used to working with, which helped in creating the realism the team was reaching for. To achieve that realism, Farquhar points to the importance of understanding the overall concept. “For example, Molly’s running around in a desert with dry, harsh ground. She's running in and out of the burrow, so when it comes to the look of the character, you look at the animation, you look at the context, and you groom with that in mind.”
As a result, Molly was given additional clumping and knots in the fur around her feet. The team reasoned that if she had been rolling around in the dirt, then it would make her fur drier, less oily, given the piece’s desert setting. “You factor that stuff into the groom to make sure that she looks believable in that environment and to give her a connection with that specific environment,” says Farquhar.
“Thelvin had done some amazing work on the shading and when we just saw Molly walking on the spot, it was really convincing,” Farquhar recalls. “That was the point I was convinced it would work.”
The shaders in Unreal Engine turned out to be very similar to those the team was used to working with, which helped in creating the realism the team was reaching for. To achieve that realism, Farquhar points to the importance of understanding the overall concept. “For example, Molly’s running around in a desert with dry, harsh ground. She's running in and out of the burrow, so when it comes to the look of the character, you look at the animation, you look at the context, and you groom with that in mind.”
As a result, Molly was given additional clumping and knots in the fur around her feet. The team reasoned that if she had been rolling around in the dirt, then it would make her fur drier, less oily, given the piece’s desert setting. “You factor that stuff into the groom to make sure that she looks believable in that environment and to give her a connection with that specific environment,” says Farquhar.
While the animation required real skill from the team, especially having the meerkat balance on the egg, the lighting also produced challenges. Lightmass was used for ambient occlusion, with indirect lighting baked using a Lightmass Importance Volume. This was coupled with dynamic lighting for the direct light, enabling the team to craft the quality of the light and produce soft bounce from the earth on the meerkat and beautifully backlit feathers on the eagle.
“In Unreal Engine, we first started with physical reality, which is what we are used to at Weta Digital—for example, making sure that we have the correct photometric lighting qualities, that the sun is around 100,000 lux, and that we are exposing the camera correctly in a way that would match reality,” explains Cabezas.
After matching reality one-to-one, the team then introduces what they call “movie lighting.” For instance, at one point Molly hides in a burrow and turns back to face the camera as the eagle attacks. For Cabezas, this was the hardest shot to light. While the lighting in Unreal Engine can be extremely realistic and physically plausible, in this case the effect was really a cheat, since in reality it would be too dark to see Molly’s facial expression.
“In Unreal Engine, we first started with physical reality, which is what we are used to at Weta Digital—for example, making sure that we have the correct photometric lighting qualities, that the sun is around 100,000 lux, and that we are exposing the camera correctly in a way that would match reality,” explains Cabezas.
After matching reality one-to-one, the team then introduces what they call “movie lighting.” For instance, at one point Molly hides in a burrow and turns back to face the camera as the eagle attacks. For Cabezas, this was the hardest shot to light. While the lighting in Unreal Engine can be extremely realistic and physically plausible, in this case the effect was really a cheat, since in reality it would be too dark to see Molly’s facial expression.
Making such cinematic lighting completely believable is a true art that sets companies like Weta apart. The meerkat is lit by indirect reddish bounce light, with subtle eye lights to signal her thinking and reaction. “The shot inside the burrow was extremely difficult; there's always that simple shot that no one really thinks about that ends up being the hardest,” says Cabezas.
To create the final piece, the team used the animation features in Sequencer, Unreal Engine’s 3D nonlinear editor. Finally, they used the Movie Render Queue to output the final frames. This feature enables you to render at maximum quality with accumulated anti-aliasing and motion blur, but slightly slower than real time (while still being orders of magnitude faster than traditional offline rendering)—ideal for linear output. The film was mastered at 1920 x 1080, as it was intended for online consumption, but the team did output some test shots at 4K, which looked stunning.
To create the final piece, the team used the animation features in Sequencer, Unreal Engine’s 3D nonlinear editor. Finally, they used the Movie Render Queue to output the final frames. This feature enables you to render at maximum quality with accumulated anti-aliasing and motion blur, but slightly slower than real time (while still being orders of magnitude faster than traditional offline rendering)—ideal for linear output. The film was mastered at 1920 x 1080, as it was intended for online consumption, but the team did output some test shots at 4K, which looked stunning.
The small team succeeded in demonstrating that in the right hands, the new standard tools in Unreal Engine 4.26 can produce Weta-quality best-of-class work—a level that exceeded the team’s own expectations at the outset of the project. In the process, they fell in love with both the characters they’d created, and with the freedom that the real-time workflows brought them.
“This project was insanely fun,” says Cabezas. “One of those experiences that at the end, you feel proud of the work that came out. Anyone in their basement can start coming up with cool little creatures, and it’s more about the creativity and the amount of skill and artistry that people can bring into it.”
“With the advances of the strand-based rendering technology in Unreal, and with off-the-shelf tools, talented artists are able to create some really beautiful work,” concludes Miller.
“This project was insanely fun,” says Cabezas. “One of those experiences that at the end, you feel proud of the work that came out. Anyone in their basement can start coming up with cool little creatures, and it’s more about the creativity and the amount of skill and artistry that people can bring into it.”
“With the advances of the strand-based rendering technology in Unreal, and with off-the-shelf tools, talented artists are able to create some really beautiful work,” concludes Miller.