by Nafisa Akabor
18 FEBRUARY 2021
Virtual reality is making the leap from the realm of entertainment and marketing to solving real-world problems
Immersive technologies, such as augmented, virtual and mixed reality, were once considered futuristic. Today we experience them.
Augmented reality (AR) comes to us though Pokémon Go or Snapchat filters and virtual reality via Google’s Cardboard, or the more expensive Sony PSVR gaming headset. And whereas AR functions in two dimensions, mixed reality (MR) blends the physical and digital worlds to allow the user to be immersed in 3D.
These technologies have been poised to change many industries, one of them being medicine, where they are used to help train surgeons. A study published by the Harvard Business Review revealed that those who trained in VR on the Osso VR platform at UCLA’s David Geffen School of Medicine received significantly higher ratings in all categories when compared with the traditionally trained group. The overall improvement in the total score was 230%.
UCLA orthopedic surgeon Gideon Blumstein says that with a strained education system in the surgical field, rapid medical innovation and a looming shortage of surgeons, VR may become an important educational tool.
Microsoft has also been testing the potential of MR in medicine. Prof Thomas Gregory of the Avicenne Teaching Hospital in France assembled a team of orthopedic surgeons from 13 countries across five continents between November and January to demonstrate how the HoloLens 2 can benefit the specialists and enable collaboration. SA was part of the experiment, the first of its kind in the country, led by Prof Stephen Roche, deputy head of orthopedics at Groote Schuur Hospital.
The HoloLens 2 headset is operated with hand gestures and voice commands. It enables surgeons to view 3D holographic images of a patient’s anatomy, created from X-rays or other scans.
The surgeries with the HoloLens 2 ranged from a knee procedure in the United Arab Emirates to a shoulder replacement in SA, with surgeons joining in to observe and offer input.
Roche assisted in a teaching capacity in three of the 13 surgeries; one locally and two in France and Germany.
He says: “In this mixed reality surgery, we can access our pre-operative planning on the X-rays or CT scan and we can adjust how we do the surgery by looking at our plans and then compare it to what we find [as we operate].”
One of the reasons the shoulder operation was chosen is because it requires 3D planning and modelling of the shoulder replacement.
“With real-time access and data planning, you can make modifications in the surgery or plan for a better outcome,” says Roche. “You can also call up articles or outcomes and discuss them with your colleagues in the theatre or people attending the surgery via a computer link.”
The HoloLens 2 has been described by Gregory as a “smartphone for surgeons”, suited for a sterile operating room.
Roche says the HoloLens can give broader access to different surgical techniques, which can be shared with universities that don’t have the ability or exposure — and in real time if necessary.
The HoloLens 2 is not available for purchase locally but will be used experimentally at Groote Schuur. Apps for it are built on Microsoft Azure, and the unit used for the holographic surgeries had bespoke apps built for this purpose.
Bronwyn Williams, a futurist at Flux Trends, says it is fantastic to see virtual reality technology making the leap from the realm of entertainment and marketing into solving real-world problems.
“The opportunities now within reach of medical science are truly extraordinary. From a medical perspective, there is huge potential for decreasing the notoriously high costs of both medical treatment and education and for increasing the efficiency and effectiveness of surgical interventions in complicated medical conditions.”
However, as a futurist, Williams says she finds two related issues worth considering as extended reality becomes more integrated into our lives, and now our bodies. “First, we need to understand that technologies like this will only accelerate the move to the future of all professions being ‘rock star’ professions — where the very best individuals in their field receive outsize recognition and rewards.
“The technology will enable the world’s top surgeons to operate on patients all across the world. Patients, given the choice, will obviously prefer to be treated by only the top physicians. This indicates a more competitive future for the next generation of doctors, and indeed, people in many other professions,” says Williams.
The second issue worth considering is the “deeper stack” involved with these technologies, specifically the software and platforms on which they run and who, or which entities, own and control them, she says.
“Platform operating systems are effectively ‘digital landlords’, and can represent a single point of control and failure — through a cyber-attack, a draconian change in terms and conditions or a denial of service. For those who depend on them, and in the case of medical devices, this could well be a matter of life and death.”
Says Williams: “As our lives become more virtualised we would do well to consider who owns and controls the substrate layer of our interface with our reality.”
