One of the challenges of training is that, until recently, the only viable way to practice has been using real aircraft, where trainees pass their examinations by performing repairs and maintenance under the watchful eye of an experienced instructor. While effective, this portion of the training can be costly and dangerous—costly due to the expense of access to aircraft and materials, and dangerous due to the trainee’s inexperience. A newbie with only classroom training, unfamiliar with the physical aspects of the work, could potentially waste materials or injure others, and could even completely ruin an airplane.
In recent years, the aviation industry has been taking a hard look at how new media like virtual reality and augmented reality can help train mechanics and ground crew more thoroughly and quickly without the cost and danger of practicing on actual aircraft. VR gives personnel a safe environment in which to hone their skills, letting them run through a maintenance scenario multiple times until the process is familiar.
One of the most recent serious players in this field is Inlusion, a USA-based VR/AR specialist company. Recently, Inlusion collaborated with aviation service companies Baltic Ground Services and FL Technics to develop new, innovative virtual reality training experiences on aircraft repair and maintenance.
The journey to XR
For the past 10 years, Inlusion has been exploring XR (cross reality), which includes VR and AR. “We experimented with XR solutions and worked on case studies in various industries like oil and gas, automotive, and aviation,” says Dainius Staras, Project Manager at Inlusion. “We wanted to figure out the potential of the technology for these industries.”After showing these projects at various conferences around the globe, the aviation industry in particular began to take notice. This led to Inlusion’s work on VR-based training with FL Technics and Baltic Ground Services.
Inlusion is an SME with around 30 developers working on real-time training solutions for the industry. Staras explains that creating a great VR training experience is much more than turning existing manuals into visual elements. “We have several team members devoted specifically to user experience,” says Staras. “This is very important in terms of creating extraordinary experiences and solving problems for businesses.”
For the past three years, Inlusion has been relying on Unreal Engine to produce their work. Staras cites the platform’s superior visual quality, along with optimization and asset management tools, as the key components behind their decision to go with UE4. “It’s easier to get better results at the end by implementing and working only with Unreal Engine in our development pipeline,” says Staras.
Simulation training for aircraft maintenance
Among the companies that were impressed with Inlusion’s projects is FL Technics, a major European player in MRO (maintenance, repair, overhaul) services for airlines.
FL Technics, in looking to stay ahead of the curve, approached Inlusion to address an industry-wide problem. With airplane mechanics in high demand, FL Technics wanted to strengthen the training for such personnel and perhaps even shorten training time. While regulations require each trainee to spend a certain number of months in training, trainees might not get to practice on certain aircraft due to lack of availability. And even if a trainee spends the required number of months, he or she might not get sufficient practical experience on every aspect of a mechanic’s job.
“There are so many regulations that mechanics have to follow,” says Staras. “For instance, if they're walking across the warehouse and they see a tool lying on the floor, they can’t just pass by it, and they also can’t just pick it up and put it somewhere. They have to take it and follow an exact procedure to document where they found it. Before they can be considered fully trained, they have to know this procedure in addition to actual aircraft repair.” However, if a trainee never happens to find a tool on the floor during the mandated training period, he or she isn’t going to get to practice this procedure.
There’s also the challenge of keeping mechanics’ skills fresh. “Suppose a plane comes in and it has a very specific defect in the engine,” says Staras. “What’s the chance that a mechanic can remember how to fix this specific defect on this specific plane if they were last trained on it two years earlier?” The reason for pursuing a VR solution, he says, is that VR gives newcomers additional practice so they can get through their mandated training faster, and also serves as a refresher for mechanics already on the job.
The result of Inlusion’s collaboration with FL Technics is a VR training system for the Boeing 737, where mechanics practice opening the thrust reverser using procedures from the aircraft’s manual.
The VR experience has two modes: one for training, and the other for examination. In training mode, the trainee is guided with visual hints, and can restart, make mistakes, or watch the task being completed automatically. In examination mode, various curveballs are thrown in to ensure the trainee can react appropriately to a variety of circumstances. The end result is a trainee with hours of practice, without the expense and danger of operating on real aircraft. FL Technics hopes that such training modules will one day reduce the mandated three-month training period for mechanics to three weeks.
While developing the VR experience, the Inlusion team came across an unforeseen challenge: during the procedure, the mechanic has to lie down and slide under the aircraft. Such a move could be awkward and uncomfortable for a user, especially if wearing a tethered headset.
After many discussions with FL Technics, Inlusion landed on the approach of lifting up the virtual aircraft for that portion of the training. “This was a very interesting solution that came out of our user experience team working together with the client,” says Staras. “It doesn’t negatively affect the training, and we ended up with the best possible user experience.”
De-icing in VR
The procedures for de-icing and anti-icing are another natural fit for aviation maintenance training in VR. These procedures require crew to spray specific quantities of liquid chemicals onto the aircraft, lifting a hoist with one arm while spraying with the other. Besides precision, speed is also a factor, since the procedure is often performed on an aircraft close to its takeoff time.
In practicing these tasks, inexperienced crew could mishandle the spraying equipment, spray too much or too little liquid, or fail to follow safety procedures. Cost can be a big issue—the price of liquid for each de-icing/anti-icing procedure is around US$2,500, and the wrong amount of liquid in the wrong place can ruin an aircraft completely.
Baltic Ground Services, a company that offers airport ground services in Lithuania and Poland, got wind of the FL Technics project and approached Inlusion with the idea of bringing de-icing/anti-icing training to VR.
In the resulting project, the instructor can start the trainee off with the most basic scenario, then add complications—bad weather, an incorrectly positioned sprayer, a truck riding into the spray zone—until the trainee can deal with many different situations. By the time the crew member uses an actual sprayer on a real airplane, he or she has had the opportunity to solve problems and build muscle memory in a variety of optimal and non-optimal scenarios.
The response from students has been overwhelmingly positive. “The first time [in VR], it was difficult to control the hoist and spray liquid on the wing at the same time, but eventually I got used to VR practice,” said one student. “When I got into the real lift cart, my hands already knew what to do.”
Instructors like VR training because it can be used anytime. “De-icing training takes place well before the start of the season, so students forgot their theoretical knowledge and had to re-learn it when they came to practice,” said one instructor. “With VR, we can apply practical knowledge on the same day after fresh theory.” Another instructor reported that the VR training also revealed knowledge gaps in experienced veterans.
Using Unreal Engine for aviation VR
When developing applications for aviation training, speed and accuracy are of high importance. The team at Inlusion found that Unreal Engine serves these needs and more. “We like having both fast prototyping in Blueprints and options to write advanced C++ code,” says Staras.The developers also appreciate the UMG (Unreal Motion Graphics UI Designer) for creating user interfaces, and the performance monitoring and optimization tools built into UE4. The system they are targeting for running these simulations is a single PC with 16 GB of RAM and an NVIDIA 1080 GTX GPU.
“But the top thing we all like is, of course, is the unmatched rendering,” says Staras. “Unreal Engine handles everything we want to see.”
Are you interested in creating simulation training for your own industry? Contact us and we’ll get that conversation started.