Unreal Engine 5 goes all-in on dynamic global illumination with Lumen
Daniel Wright, Engineering Fellow: Graphics and Krzysztof Narkowicz, Technical Director: Graphics |
May 27, 2022Hi, we're Daniel and Krzysztof, engineers working on Lumen. Lumen is Unreal Engine 5's fully dynamic global illumination and reflections system, which is enabled out of the box. It is designed for next-generation consoles and high-end visualizations beyond games like architectural visualization. Here, we’ll walk through Lumen’s features and give an overview of the technical details. For a complete reference, see the Lumen documentation.
In the past, global illumination, for most games, had to be solved in an offline process called lightmap baking, because it was too computationally expensive to be calculated in real time. In Unreal Engine, lightmaps are baked through CPU Lightmass or GPU Lightmass. Static lighting from lightmaps can provide very high quality, but requires long build times, and greatly constrains the game environment. Any action which changes the indirect lighting in a significant way, like moving a wall-mounted television, will leave lighting in an incorrect state.
Global illumination
When light leaves a light source, it illuminates all of the surfaces visible from that light source; this is known as direct lighting in computer graphics. In real life, it doesn’t stop there though, it bounces off the surface, picking up the color of the surface as it goes. Light that bounces off of a rough surface in all directions is called diffuse indirect lighting or global illumination. Light that bounces specularly off a smooth surface is called a reflection. Eventually the light reaches your eye, or a camera, and forms an image.In the past, global illumination, for most games, had to be solved in an offline process called lightmap baking, because it was too computationally expensive to be calculated in real time. In Unreal Engine, lightmaps are baked through CPU Lightmass or GPU Lightmass. Static lighting from lightmaps can provide very high quality, but requires long build times, and greatly constrains the game environment. Any action which changes the indirect lighting in a significant way, like moving a wall-mounted television, will leave lighting in an incorrect state.
Games that required these dynamic scenes have been relying on low-quality irradiance probe-based lighting, and approximations like ambient occlusion. Global illumination that rivals baked lighting in quality has remained elusive.
Lumen simulates light bouncing around the scene in real time, enabling players to change any aspect of the game world, with indirect lighting automatically updating. Players can destroy large parts of levels, change the time of day, or flood a portion of the level and those changes will automatically propagate to the lighting. Game developers can say goodbye to seeing the ‘LIGHTING NEEDS TO BE REBUILT’ message in the Unreal Editor.
Lumen simulates light bouncing around the scene in real time, enabling players to change any aspect of the game world, with indirect lighting automatically updating. Players can destroy large parts of levels, change the time of day, or flood a portion of the level and those changes will automatically propagate to the lighting. Game developers can say goodbye to seeing the ‘LIGHTING NEEDS TO BE REBUILT’ message in the Unreal Editor.
Lumen solves global illumination at high quality, including effects like color bleeding and indirect shadows. Lumen supports infinite diffuse bounces, which are important in scenes with bright surfaces like the white paint in the scene below.
Rafael Reis of UE4Arch
While Lumen is solving global illumination, it also solves sky shadowing, which causes indoor environments to be darker than outdoor ones.
Lumen also propagates lighting from emissive materials like the lamps and windows below, without any additional cost. There's a limit to how small and bright emissive areas can be before they cause noise artifacts to appear. Emissive lighting is inherently much more difficult to solve than manually placed light sources.
Lumen also solves global illumination on translucency and volumetric fog, although at lower quality. Here, the emissive sign mesh is dynamically lighting up the volumetric fog.
Reflections
Lumen solves reflections dynamically, off any surface, with any roughness. This replaces precomputed forms of reflections like reflection captures, and limited techniques like planar reflections and screen space reflections.
It’s not enough to solve dynamic global illumination for what’s directly visible from the camera, though; Lumen also provides dynamic GI and shadowed skylight seen in reflections.
Model courtesy of Audi
Lumen reflections support coated materials, like this car paint, which has a mirror reflection off the outer coat, while having a glossier reflection off the interior layer.
Model courtesy of Audi
Ray tracing in Lumen
Lumen uses ray tracing to solve these lighting features, and uses an approximate version of the scene for faster ray intersections. This scene can be visualized by switching to the Lumen Scene View Mode. This view shows what Lumen ‘sees’ when it traces rays for global illumination and reflections.
By default, Lumen uses Mesh Distance Fields, a technique called software ray tracing, because it doesn’t require a video card with hardware support for ray tracing. Distance fields represent the surface of a mesh in a way that’s very fast to intersect rays with.
Software ray tracing also merges meshes into a Global Distance Field, which provides very fast ray intersections, even when there are many overlapping meshes—often the case with hand-placed Megascans. Epic’s technology demos Lumen in the Land of Nanite and Valley of the Ancients used Global Distance Field tracing exclusively because of the heavy amount of mesh overlap.
Lumen can also use hardware ray tracing through a project setting. Hardware ray tracing is more accurate but more expensive, with more geometry types represented, including skinned meshes. Hardware ray tracing is required for mirror reflections.
Rafael Reis of UE4Arch
In The Matrix Awakens: An Unreal Engine 5 Experience technology demo, the tall city buildings block the direct lighting from the sun, leaving Lumen to provide lighting for most of the screen.
Lumen used hardware ray tracing in this demo for high-quality reflections, skinned meshes, and huge view ranges.
Lumen begins populating the Surface Cache by choosing which directions to capture each mesh from, such that all of its surfaces will be covered. As the player moves through the world, Lumen is constantly recapturing the material attributes of nearby meshes into the Surface Cache atlas. The Nanite virtualized geometry system greatly accelerates these mesh renders, especially on high-polycount meshes, although it is not required for Lumen. These surfaces are then lit, including computing multi-bounce global illumination and shadowed skylighting, which will be seen in reflections. The Surface Cache lighting is now ready to be used to accelerate features like global illumination and reflections.
Surface caching
When a ray hits a surface, Lumen needs to evaluate the material and the incoming lighting. This is very expensive to do at the end of each ray, and inefficient; multiple rays may hit the same point, and the material may have many texture lookups. Lumen caches this lighting in its Surface Cache, which is enabled by default.Lumen begins populating the Surface Cache by choosing which directions to capture each mesh from, such that all of its surfaces will be covered. As the player moves through the world, Lumen is constantly recapturing the material attributes of nearby meshes into the Surface Cache atlas. The Nanite virtualized geometry system greatly accelerates these mesh renders, especially on high-polycount meshes, although it is not required for Lumen. These surfaces are then lit, including computing multi-bounce global illumination and shadowed skylighting, which will be seen in reflections. The Surface Cache lighting is now ready to be used to accelerate features like global illumination and reflections.
The Surface Cache is a key optimization of Lumen, but also has content considerations. In particular, only meshes with simple interiors can be supported—walls, floors, and ceilings should all be separate meshes. Importing an entire room that includes furniture as a single mesh is not expected to work with Lumen.
The Surface Cache View Mode shows pink where there is no coverage. These areas will appear black in reflections, and will cause view-dependent global illumination, as only Lumen screen traces will be active.
Final gather and denoising
Ray tracing is a very expensive operation, so we can’t afford to trace very many rays. At the same time, every pixel on the screen needs to consider receiving lighting from every other surface in the scene; that’s why it’s called ‘global’ illumination. We can’t possibly trace rays in all directions, so we have to pick a small subset. If we miss important directions, it will show up as noise artifacts in the final image, so we have to be smart about which directions we trace rays in, and try to reuse that small number of traces wherever possible.For diffuse global illumination, Lumen uses an advanced final gather algorithm based on radiance caching, which was presented at Siggraph 2021.
Lumen’s final gather uses a screen-space radiance cache to heavily downsample the global illumination calculation from the main screen. The downsampled indirect lighting is integrated with the full-resolution material data to produce full-resolution shading.
The incoming lighting is filtered within this radiance cache, which greatly smooths out noise and reuses the precious traces between neighboring pixels. This is different from a screen-space denoiser, as we’re still operating in the downsampled radiance cache space, which is much more effective.
Lumen chooses which directions to trace its rays very carefully; it checks in directions that had bright lighting last frame, a technique known as importance sampling of incoming lighting. Lighting from the previous frame is used to guide the search for lighting in this frame, which gives the same quality as if we had traced four times more rays, but runs much faster. In the screenshot below, the white rays were chosen using importance sampling.
Finally, Lumen separates out nearby lighting from distant lighting, and traces many more rays to resolve distant lighting to reduce noise, through a technique called world-space radiance caching. This becomes very important when an indoor room is lit entirely by the sky lighting entering a small window, and is key to Lumen’s indoor quality.
For reflections on rough surfaces, Lumen reuses the screen-space radiance cache at no extra cost. This is also how Lumen is able to provide a second layer of reflections on clear coat materials like car paint without doubling the cost.
Rafael Reis of UE4Arch
Reflections on smooth surfaces trace additional rays to find the reflected light. The lighting found by these rays is shared between neighboring pixels and temporally accumulated across frames to smooth out noise.
All of these sampling and denoising techniques together are critical to Lumen’s quality and performance. Lumen wasn’t designed for direct lighting, but it is sometimes possible to light scenes entirely with emissive meshes thanks to the final gather quality. In The Matrix Awakens: An Unreal Engine 5 Experience, the experimental night mode uses only Lumen global illumination and reflections to propagate the light from millions of emissive window meshes; no light sources were placed by artists.
Performance and quality
Lumen relies heavily on temporal upsampling with Unreal Engine 5’s Temporal Super Resolution algorithm for 4K output. This gives the best final image quality, instead of running Lumen at 4K natively with significantly lower quality settings.Lumen can achieve 60 FPS on next-generation consoles when the engine is using the high scalability level for global illumination and reflections, however quality with these settings is still a work in progress.
In the other direction, Lumen can scale up in quality with:
- Increasing Final Gather Quality in the post-process volume settings
- Using hardware ray tracing
- Using hit lighting when the rays hit surfaces for higher quality instead of surface caching
Rafael Reis of UE4Arch
Final words
We hope you enjoyed this summary of Lumen’s capabilities and inner workings. Lumen is still under development and we are working hard to improve it. In particular, we are working toward better performance and quality when running within a 60fps budget on next generation consoles, and expanding its use cases to automotive, virtual production, and movie rendering.Download Unreal Engine 5.0 today!
If you’re an existing Unreal Engine user, you can download Unreal Engine 5.0 from the Epic Games launcher. If you're looking to dive in for the first time, click the link below to get started. Either way, we hope you enjoy all the new features and upgrades, and as always, we encourage your feedback!