Rivian’s R1T features state-of-the-art HMI, designed, developed, and deployed in Unreal Engine

January 24, 2022

Electric Adventure Vehicle maker Rivian is not your average automotive manufacturer. On a mission to “Keep the World Adventurous Forever,” the company is driven to build the kind of future “our kids’ kids’ kids deserve.”

As staggering and complex as this objective sounds, Rivian sees every challenge as an opportunity to seek out better ways of doing things. That’s why it used Unreal Engine to build a state-of-the-art HMI (Human-Machine Interface) for its brand-new R1T—the first fully electric pickup truck to be delivered to the consumer market.

Choosing a real-time solution for HMI development and deployment

Rivian evaluated a number of solutions before choosing Unreal Engine. “We put it through an entire workload that simulates driving through traffic,” says Eddy Reyes, Staff Software Engineer, In-Vehicle Experience, at Rivian. “We wanted to make sure that Unreal Engine running on our hardware, in a driver-assistance workload showing multiple cars on screen, would perform really well.”

The results, reveals Reyes, showed that Unreal Engine was not only the most performant, but also the best looking. “Unreal Engine just blew all of the other options out of the water, and it had the best graphics, it had the highest quality,” he says. “It was an easy decision.”

The next key milestone was to deploy Unreal Engine onto a vehicle equipped with sensors. The team could see all the data picked up by the sensors being rendered on screen in real time as they drove the vehicle. “That was the second time we celebrated,” says Reyes. “It was crazy.”

Unreal Engine just blew all of the other options out of the water, and it had the best graphics, it had the highest quality. It was an easy decision.

”

- Eddy Reyes, Staff Software Engineer, In-Vehicle Experience, Rivian

Matt Metropulos, Design Director of Vehicle UI at Rivian, describes how this sensor data feeds into the real-time rendered graphics. “At this early stage of autonomy, our drivers need to build trust with the vehicle. Unreal Engine allows us to visualize the vehicle reacting to real-time situations on the driver display. We do this in concert with the vehicle’s exterior sensor system.”

All this was happening on a very accelerated timeline. “I think the main advantage Unreal Engine brought us is really velocity from a development standpoint.” says Wassym Bensaid, Vice President of Software Development. “Once we have the 3D representation of the car, the speed of development really allows us to enable more and more use cases in a very fast way.”

One of the things that contributed to the pace of development was Blueprint, Unreal Engine’s visual scripting. It enables developers to quickly prototype and iterate on advanced effects and behaviors without ever writing a line of code.

Reyes describes how the team used the system to iterate on the drawing of the lanes on screen to represent the road curvature. “There's a lot of math involved to convert the data that you receive from the cameras into the engine,” he says. “We had to go through multiple iterations until we got it right. We couldn't have figured that out if we weren't using Blueprint to prototype.”

A collaborative platform for engineering and design

Beyond just technically getting the data onto the screen, there are many challenges in building a safe and effective HMI. “You don't want to distract people,” explains Eric Wood, Vice President of User Experience. “You don't want to give them too much information, too little information. There is occlusion [...] There's glare […] So you just can't start with a graphic design and say, ‘this is really cool,’ and then make it a driving interface—it's so much more complex than that.”

That’s where the collaboration between the engineering and design teams came in. “It took a lot of iterations,” says Wood. “How much was too much? Are we over-warning people? Really quickly, we dialed it in using Unreal Engine. I had designers working on animations, working on behaviors, showing it to the engineers, which are like two doors down, them coming up with something and then saying, ‘Can you come check this out?’ And that was just so exciting, that you could go from concept to trying it out in a day or two.”

Having both the design team and the software team working within Unreal Engine creates efficiencies, explains Metropulos. “It allowed us to develop a shorthand, so the team could quickly problem-solve together,” he says. “This also enabled the team to be more accountable to each other and therefore more autonomous in managing work.”

Reyes agrees. "I like the fact that we can bring designers and engineers into the same platform and have them exchange ideas every day on just Unreal Engine,” he says. “We’ve been able to build a whole team around this particular engine, just the skills that people are learning and bringing to the table to make the content better,” says Reyes. “It's really cool, because now, there's a common platform that everyone can come together to add their contributions.”

High-fidelity vehicle representation

One of the things that makes the R1T’s HMI so engaging is the fidelity between the actual vehicle and its on-screen representation. This is an important factor in building trust with the user.

“Unreal Engine enables us to customize the rendered vehicle based on its configuration,” says Metropulos. “So anywhere within our interface, we have flexibility to show the vehicle in its specific color, wheel, and trim. We can also show the real-time status of the vehicle’s doors, tonneau cover, or Gear Tunnel.”

Because Unreal Engine can handle all of the R1T system’s polygon budget (about 500,000), Rivian was able to use data from the original CAD model of the vehicle, rather than having to rebuild it from scratch. This not only saved time, but made the representation very accurate, as Wood explains. “It's not a simulation of one of our vehicles—it's actually our vehicle that's running there in your driver display in real time,” he says.

Room for growth and over-the-air updates

Another advantage of Rivian’s implementation is that it allows for over-the-air (OTA) updates, enabling Rivian to deliver new content to its customers. “Whether it's in the driver display or in the center display, for safety or for entertainment—all of that headroom that Unreal affords us is a real gift," says Wood. “The stuff that we are now imagining, it's something that you will have never seen in a vehicle before, and that's exciting to me.”

The Unreal Engine HMI Design Challenge

If you’re excited about HMI development, whether or not you have any previous experience, why not enter our HMI Design Challenge? Created in partnership with Car Design News (CDN), this competition asks designers to rethink what’s possible with real-time technology. You can find all the details and sign up here.

Accelerate your HMI development

Interested in finding out more about using Unreal Engine for HMI design, development, and deployment? Visit our dedicated hub.

Visit the hub