Bringing Nanite to Fortnite Battle Royale in Chapter 4
Graham Wihlidal, Engineering Fellow, Graphics |
January 26, 2023Hello, I’m Graham Wihlidal, Engineering Fellow (Graphics) at Epic Games. I’m here to showcase the exciting features and improvements we’ve developed this year in order to launch Nanite in Fortnite Battle Royale Chapter 4. These features are available for you to try out in Unreal Engine 5.1 (as Beta).
Unreal Engine 5.0 launched with Nanite in a production-ready state. It was already usable for tons of amazing things, but the initial version lacked full support for all the myriad of features available for non-Nanite meshes. Instead, we’d focused our time on polishing the core Nanite functionality.
Going forward, we’d like to extend support into many areas Nanite has not yet covered. Users have been heavily requesting support for features like World Position Offset, Pixel Depth Offset, custom UVs, two-sided materials, and masked materials. At the beginning of the year, the team undertook the challenge of trying to implement support for some of these features, requiring us to solve numerous non-trivial problems.
GPUs started with what was known as a fixed function pipeline, where the method in how geometry was transformed—and how depth and color was written—was built into the hardware, and could only be configured with a limited set of pre-defined functions. Later on, hardware became “programmable” through shader code, enabling new possibilities for graphics and allowing for features that were difficult or impossible to do using the fixed function pipeline.
From its inception, Nanite has always supported “programmable” material graph shaders controlling the output color, but the rasterizer itself—which decides how vertices are positioned on screen and which pixels get covered by the triangles—was effectively “fixed function”:it was still implemented as shader code, but a content creator had no control over that logic.
In order to support the aforementioned features in Nanite, we needed to make the rasterizer itself programmable.
The following images show the initial prototype of a masked material animating over a Nanite garbage truck mesh.
Unreal Engine 5.0 launched with Nanite in a production-ready state. It was already usable for tons of amazing things, but the initial version lacked full support for all the myriad of features available for non-Nanite meshes. Instead, we’d focused our time on polishing the core Nanite functionality.
Going forward, we’d like to extend support into many areas Nanite has not yet covered. Users have been heavily requesting support for features like World Position Offset, Pixel Depth Offset, custom UVs, two-sided materials, and masked materials. At the beginning of the year, the team undertook the challenge of trying to implement support for some of these features, requiring us to solve numerous non-trivial problems.
GPUs started with what was known as a fixed function pipeline, where the method in how geometry was transformed—and how depth and color was written—was built into the hardware, and could only be configured with a limited set of pre-defined functions. Later on, hardware became “programmable” through shader code, enabling new possibilities for graphics and allowing for features that were difficult or impossible to do using the fixed function pipeline.
From its inception, Nanite has always supported “programmable” material graph shaders controlling the output color, but the rasterizer itself—which decides how vertices are positioned on screen and which pixels get covered by the triangles—was effectively “fixed function”:it was still implemented as shader code, but a content creator had no control over that logic.
In order to support the aforementioned features in Nanite, we needed to make the rasterizer itself programmable.
Initial prototype
We set out to prototype the architecture needed to support material graph logic in the rasterizer, which has been coined “Nanite Programmable Rasterizer.”The following images show the initial prototype of a masked material animating over a Nanite garbage truck mesh.
The prototype was successful in proving out whether or not the programmable rasterizer would even be possible, but lots of work remained to make it production-ready and efficient.
We defined some clear goals to drive the development of this work:
The initial prototype was hardcoded to only support a single programmable material in the scene, so the next step was a proper rasterizer “binning” pass that would let us support actual game scenes composed of hundreds of materials, which then enabled us to test out actual content.
We defined some clear goals to drive the development of this work:
- Keep the performance profile of the existing “fixed function” fast path—the introduction of the programmable rasterizer should not slow down existing content.
- Ensure that fixed function and programmable rasterizer paths largely share the same code path, for maintainability reasons.
- Bear in mind that the programmable rasterizer will be used heavily by a lot of content, so simple evaluators should deliver performance that is only marginally slower than the fixed function rasterizer.
- Only perform the instance culling and cluster culling work once.
- Minimize additional memory impact to GPU scene and Nanite.
- Deliver this as a production feature in UE 5.1.
The initial prototype was hardcoded to only support a single programmable material in the scene, so the next step was a proper rasterizer “binning” pass that would let us support actual game scenes composed of hundreds of materials, which then enabled us to test out actual content.
Medieval Game shot converted almost entirely to Nanite!
Nanite triangle visualization
Unique rasterizer “bins” (materials) in the scene
Even with the Medieval Game test scene functional, there was a lot of remaining optimization and feature work to make the Nanite Programmable Rasterizer framework shippable in a game.
Very early on, it became clear that there was a desire for Fortnite Battle Royale Chapter 4 to leverage Nanite, but we did not yet support the necessary features needed for wholesale adoption. Even the seemingly simple meshes like opaque building pieces needed World Position Offset to animate the iconic “bounce” effect when damaged. Fortnite became the first user of the programmable rasterizer.
Very early on, it became clear that there was a desire for Fortnite Battle Royale Chapter 4 to leverage Nanite, but we did not yet support the necessary features needed for wholesale adoption. Even the seemingly simple meshes like opaque building pieces needed World Position Offset to animate the iconic “bounce” effect when damaged. Fortnite became the first user of the programmable rasterizer.
Fortnite use cases
Animated props
The first use case to support is simple opaque static mesh props that use World Position Offset for secondary animation. While these props do not require Nanite to render efficiently, Virtual Shadow Map performance scales much better with Nanite meshes, so it was important to render as much of the scene using Nanite as possible.
Buildings
A major aspect of Fortnite is the various building sets, and Nanite has already been proven to handle large cities with ease. It should be no surprise that we chose to use Nanite for all the building meshes—to increase visual fidelity, eliminate level-of-detail popping, and for better performance.
Construction
There are far too many meshes of buildings in Fortnite to rebuild them all from scratch across all platforms, so we created an offline process that can take artist-authored displacement textures and rules, and produce high-quality displaced Nanite meshes with much higher triangle count than the traditional versions.Note: This is not “runtime” displacement, it is a Fortnite-specific offline workflow for displaced mesh creation.
In addition to visual improvements to the primary view rendering, the displaced Nanite meshes also greatly improved the Virtual Shadow Map quality, since geometric detail like bricks— previously represented as 2D painted areas on the texture—are now displaced into actual 3D space. This gives the surfaces increased depth and detail, allowing for proper self-shadowing and silhouettes.
The buildings in Fortnite are all opaque static meshes and could be rendered entirely with our Nanite feature set available in Unreal Engine 5.0, except for the “wobble” effect that occurs temporarily while a building piece is taking damage (such as a player hitting it with a pickaxe).
The wobble is a simple animated track applied through World Position Offset in the material, and the evaluation is always being performed, even when the wobble is not visually occurring (a zero weight is used).
With the significant number of building meshes in Fortnite, we implemented an optimization to target this pattern where a special mode can be enabled (r.OptimizedWPO), and then regardless of whether or not a material has logic to drive World Position Offset, Nanite will only evaluate this logic if a given primitive component has “Evaluate World Position Offset” enabled (which is the default setting).
The wobble is a simple animated track applied through World Position Offset in the material, and the evaluation is always being performed, even when the wobble is not visually occurring (a zero weight is used).
With the significant number of building meshes in Fortnite, we implemented an optimization to target this pattern where a special mode can be enabled (r.OptimizedWPO), and then regardless of whether or not a material has logic to drive World Position Offset, Nanite will only evaluate this logic if a given primitive component has “Evaluate World Position Offset” enabled (which is the default setting).
When Nanite performs the aforementioned “rasterizer binning” pass, any primitives that normally would have taken the programmable path but that have “Evaluate World Position Offset” disabled will instead take the standard fixed function rasterizer path.
This optimization has proven useful in a number of places (including disabling World Position Offset in the distance), but it was vital for the building wobble. We disabled “Evaluate World Position Offset” by default on all building meshes, and adjusted the game code in Fortnite to programmatically set this value based on whether or not the building was currently being damaged (and wobbling).
This optimization has proven useful in a number of places (including disabling World Position Offset in the distance), but it was vital for the building wobble. We disabled “Evaluate World Position Offset” by default on all building meshes, and adjusted the game code in Fortnite to programmatically set this value based on whether or not the building was currently being damaged (and wobbling).
Along with this new optimization, we added an “Evaluate WPO” Nanite debug view (r.Nanite.Visualize EvaluateWPO) that shows green if a mesh is currently evaluating World Position Offset and red otherwise.
With this optimization, almost all building meshes will take the fixed function path, except for just a handful of meshes occasionally taking the programmable wobble path when necessary.
Example shot
r.OptimizedWPO off vs on
Trees
The area we spent the most prototyping and development time on was the trees. For Chapter 4, we wanted lush forest areas, so we needed an efficient solution with predictable performance. The trees relied on entirely brand-new functionality in Nanite that we had never shipped in a title before, so a lot of prototyping and optimization work took place to determine the approach we would ultimately use.
Construction
Our initial experiments were using masked materials and cards for trees.
For Fortnite’s content, we found it was usually faster to avoid masked materials and instead rely on increasing triangle counts of the mesh and keeping the materials opaque, especially for trees and grass. This is mainly because masked materials in Nanite incur a significant cost during base pass shading, since we have to recalculate triangle barycentrics per pixel, and negative space in the alpha maps adds additional overdraw cost.
Preserve Area
After converting Fortnite trees to Nanite, we noticed they would lose their canopies in the distance due to the simplification process; either the leaves would thin out, or in some cases, abruptly go bare. At some point, each individual leaf could not simplify any further than a single triangle and leaves needed to be removed to reduce triangle count. Removing leaves like this causes the canopy to visually thin out.To correct that, we added new logic to the Nanite builder (an option called “Preserve Area” that can be enabled under mesh “Nanite Settings”) that redistributes the lost area to the remaining triangles by dilating out the open boundary edges. In the case of leaves, this is the same as the remaining leaves getting bigger. While that looks odd if performed up close, at the distance when area preservation activates, it instead appears to maintain the density that should be there.
This feature should only be enabled on foliage meshes exhibiting this problem, and on nothing else.
Without Preserve Area
With Preserve Area
Wind animation
It would look odd if all the trees rendered at high fidelity but did not animate in the wind. Traditionally, Fortnite would accomplish wind animation with complex logic driving World Position Offset. With the Nanite trees having significantly more vertices (~300-500k) than their non-Nanite counterparts (~10-20k), and with the World Position logic now being evaluated inside of Nanite’s rasterizer, we explored other ways to animate the trees in a manner that reduced the per-vertex evaluation cost.
Our experiments led to baking a complex wind simulation into a texture that we could sample when driving World Position Offset, rather than evaluating a ton of complex math for every vertex.
From the tree geometry, we can extract the pivot of each branch and the level within the hierarchy, as well as each branch’s parent branch. We can make a skeleton from this information and input it to a Houdini Vellum simulation.
Each branch’s pivot and orientation from the simulation can be encoded as a pixel value into an image, sampled in a material shader, and indexed using a custom UV value encoded into the mesh asset that is used to pick the appropriate row of pixels for the branch.
Because of this, the Nanite rasterizer only needs to look up a single position and quaternion to calculate the offset, and does not rely on multiple dependent texture reads. This approach currently only supports rigid animation, as each branch has the same UV value.
Because of this, the Nanite rasterizer only needs to look up a single position and quaternion to calculate the offset, and does not rely on multiple dependent texture reads. This approach currently only supports rigid animation, as each branch has the same UV value.
Distance culling
The wind simulation on trees is very important close to the camera but fairly indistinguishable out in the distance. As a performance optimization, we added the ability for Nanite to disable World Position Offset calculations at an artist-defined distance. This optimization builds on top of the “Evaluate World Position Offset” mode.
Grass
We added support for Landscape grass to spawn Nanite mesh instances, including support for distance culling inherent to this system. The asset approach taken here is similar to the approach for trees. We used opaque materials with actual geometry for the blades of grass but just simple math driving World Position Offset animation.Landscape (Nanite)
Due to the nature of how Virtual Shadow Maps work, large non-Nanite meshes render shadows much slower than a Nanite mesh of equal size. This was especially problematic for Landscape, which is a massive non-Nanite mesh that was costing a lot of GPU time. We implemented an experimental Nanite rendering feature for Landscape, released in UE 5.1, that converts the landscape height field into a Nanite mesh at build time. The conversion does not increase visual quality, but provides all of the performance characteristics of Nanite culling and rasterization when rendering into the base pass or Virtual Shadow Maps.
Lumen has a specialized path for tracing against the Landscape height field, and Nanite does not support rendering into Runtime Virtual Textures at this time, so in those cases the Nanite representation is not used, and the height field is used instead.
Future work
Perhaps the most important remaining thing is accurate (per-frame) instance and cluster bounding volume computation that matches the animation performed by World Position Offset. This is vital to ensure that Nanite’s occlusion culling removes as many clusters as possible before rasterization.At the moment, we use the reference pose bounds (ignoring World Position Offset), which is either overly conservative (rasterizing more clusters than needed), or causes visual artifacts if the World Position Offset animates clusters fully outside of the reference pose bounds (pieces of the mesh disappear). We implemented rough support for Bounds Scale on primitive components (similar to how non-Nanite approaches this problem), where you can scale up the bounds by some arbitrary amount, but this makes the culling even more conservative, costing unnecessary performance.
We also have plans to continue optimizing the material system in Nanite to reduce the cost of masked materials and materials with Pixel Depth Offset.
We’re excited to release this new functionality for Nanite in Unreal Engine 5.1 and look forward to seeing all the amazing content developers create with it!
For more information on the features referenced in this blog, visit the Nanite documentation.
Get Unreal Engine today!
Get the world’s most open and advanced creation tool.
With every feature and full source code access included, Unreal Engine comes fully loaded out of the box.
With every feature and full source code access included, Unreal Engine comes fully loaded out of the box.