March 7, 2020
Precision OS delivers accredited curriculum for orthopedic surgical training in VR
A very random meeting
Formed in a Vancouver basement two years ago, Precision OS is now a flourishing business that provides orthopedic surgical training in virtual reality. Two of its three founders, CTO Colin O’Connor and Chief Creative Officer Roberto Oliveira, had both worked in the video game industry for decades. After years at Radical Entertainment, Black Box Games, and industry giant Electronic Arts they helped co-found United Front Games together, where they had critical and commercial success with titles like ModNation Racers and Sleeping Dogs.In 2016, the pair were looking for something new to get their teeth into when they had what Oliveira describes as “a very random meeting” with orthopedic surgeon Dr. Danny Goel, now CEO of Precision OS. After a get-together at a local pub which included a demo from O’Connor of the newly released HTC Vive, the team started building a VR training platform for orthopedic surgery. Initial feedback from surgeons, residents, and device companies was positive, and the three went all-in and started the business.
Rethinking surgical training
Traditionally, surgeons have trained using plastic models and actual cadavers. As Goel—who continues to practice as a surgeon—explains, neither of these “simulates” real conditions accurately enough. In the case of cadavers, the condition of the specimen can negatively impact the experience; although variation is at the heart of medicine, the actual core of the simulation should be consistent. Similarly, plastic models lack the contextual aspects of real-life surgery. The same is true of medical reference books, which can’t deliver the experiential nature of surgery.“The interesting part about anatomy is when you look in one of these medical books, everything's clean and broken apart and you see bones that are white and muscles that are very well defined,” says Oliveira. “And then when you go into surgery and you look into the approach or the incision, it's completely different than what you would expect.”
For Precision OS’s VR training, the goal is to simulate the real environment as closely as possible, enabling students to experience what surgery looks and feels like. They encourage trainees to make mistakes in simulation—without putting patients at risk. To familiarize themselves with the procedures and thereby recreate them as closely as possible, Oliveira and O’Connor physically stand behind Goel and watch him operate. They also study medical books and actual cadavers.
And then it’s a case of recreating what they have witnessed in a real-time VR environment. Having had experience with several game engines in the past, including writing their own at United Front, Oliveira and O’Connor chose Unreal Engine.
“I know for a fact that Unreal supports more things out of the box than any other engine out there,” says O’Connor, explaining the choice. “And I wanted to make sure that we hit that triple-A fidelity mark right from the outset.”
With his rendering engineering background, the openness of the Unreal Engine platform was also a key factor for O’Connor. “I also wanted to make sure that I had access to the code and could go right down to the hardware layer and internal GPU submission calls, to edge every single bit of performance out of the VR experience,” he says.
Recreating operating theater reality
To create the virtual patient, the team initially purchased an anatomy model set, but quickly found the limitations of that, so they started modeling their own. Recently, in a bid to increase the authenticity, they had a patient scanned. In selecting their candidate, they ensured that, unlike the stock model, he was of the typical age and physique of someone needing the operation—that is to say, an older person with a common body habitus.The accuracy of the simulation is particularly important when dealing with surgery, where a misrepresentation could have significantly grave consequences. Oliveira and O’Connor use every trick they’ve ever learned in game development to make sure they are able to represent each step as faithfully as possible, and Unreal Engine’s deep and broad feature set is part of the solution.
“With the power behind the Blueprint system, the animation system, support for morph targets, vertex animation...Unreal just gives us an array of technology that we can hook into to solve these problems of recreating a medical environment,” says O’Connor.
“There are ethical considerations to what we are building,” says Goel. “Misrepresentations and over-optimism of VR are critical elements when creating something with consequences to actual patients. We are sensitive to both and are researching all aspects of virtual reality. A second and important element to also consider is the point about empathy. It is important for the trainees to remember how their practice has implications to patient lives.”
Teaching illustrative anatomy
Goel also wants to ensure that the module teaches anatomy illustratively, learning he values in his own work on the operating table.“During surgery, although you can't see each and every muscle, an experienced surgeon understands the anatomical landscape very well,” he says, “During surgery, I see surgical anatomy but I am thinking about illustrative anatomy. Although you never fully see certain nerves, vessels, and muscles, in my mind I'm imagining where these structures are, to avoid a misplaced retractor or inadvertent injury.
“Anatomy is a three-dimensional concept that we learn in two dimensions. Unless I have regular unencumbered access to a cadaver, how do I learn and reinforce my anatomical learning? This point is really important to us here at Precision OS, where we focus on recreating as much of the realistic illustrative anatomy as possible in 3D. Having a deep understanding and appreciation for anatomy is the foundation of surgery in all specialties.”
Beyond the visual
As well as the visual aspect, the application features auditory feedback, so that you can hear the anesthetic machine, or the sound of a drill or mallet as you use it. And haptics are also employed, but only where they are critical to support the training. Goel explains that it’s the ability to make decisions during the training, and to make mistakes, that forms their double-loop simulation experience inherent in what Anders Ericsson has coined “deliberate practice”.“The decisions you make prior to and during surgery are how we impact patient outcomes,” he says. “This decision-making process is what we embed within our simulation modules.”
Selecting the hardware
When discussing haptics, both the cost and the portability of the hardware required to support them are also factors in the extent to which they are used. Many of their customers travel with the educational gear, and need to be able to quickly set it up and tear it down. And the team is keen to decrease the disparity in health care that exists in different parts of the world.“Adding more complex hardware restricts who could have access,” says Goel. “Impacting the health care disparity that exists in certain parts of the world is a major consideration for us at Precision OS. We have therefore maintained our focus and dedication to creating the most impactful educational software while using the most portable hardware, permitting global distribution.”
Increased portability is also a goal so that student doctors can consume the training at their convenience, in their own home, office, or school. Currently the hardware implementation is a laptop tethered to a headset, which adds an element of friction for transportation, but the team is moving to mobile VR devices and devices like the Oculus Quest within the next year.
Training the next generation of surgeons
So has there been resistance to using this kind of technology for training?“It's quite interesting,” says Oliveira. “We see slight resistance sometimes where we don't expect to see it, like in younger students or younger doctors, and then sometimes we expect to see it in the older generation and sometimes we get an incredible reaction there. We’ve never really had to push the technology. People seem to understand this is the future of surgical training. Most organizations are just trying to figure out the best way to introduce it.”
Overall, as the technology becomes more affordable, accessible, and portable, it is seeing wide acceptance. The fact that the system can also be used to collect performance data and provide metrics for the students on the backend is another clear benefit to those whose mandate it is to train and educate the next generation of surgeons.
Today, Precision OS modules are in use by hundreds of residents in the ten North American universities and institutions that were their original partners, and, in conjunction with their other customers, are also available in in countries as far apart as Japan, Switzerland, France, and Australia. With their product for the international organization known as the AO Foundation and a new preoperative planning tool, they plan to educate thousands of people from North America, and then tens of thousands globally.
In May of 2019, Precision OS received accreditation from a provider to the Royal College of Physicians and Surgeons of Canada, enabling their training to be used as the performance appraisal component of continuing medical education (CME) for surgeons. This accreditation, coming as it does from a highly regarded organization, is a validation of the company’s efforts to achieve the highest-quality training through VR.
The marriage of medicine and technology is disrupting the well-entrenched methods to educate physicians and surgeons. Understanding this, the whole team at Precision OS creates content with a heightened sense of social responsibility. “As an operating surgeon, I know the time and energy we spend on the details above could have significant implications,” says Goel. “What and how the trainees learn is of the utmost importance to us. The trust our partners and users have placed in us is reflected in our content for a single reason—for them to practice with purpose in virtual reality, so they may operate with precision, and truly impact patient care worldwide.”
Interested in finding out how you could use Unreal Engine 4? Get in touch and we’d love to start that conversation.