How a dose of virtual reality could change healthcare in the next decade
David Proudlock, Innovation Consultant at the Pistoia Alliance shares his experience of judging the Virtual Medicine Conference Virtual Reality Hackathon and explores the potential of AR/VR in life science and digital healthcare
The advent of extended, augmented or virtual reality has seemingly been on the cusp of mainstream adoption for many years, there are plenty of examples of its use, but it's still not common to see someone in the street using these technologies and the same could be said within the workplace or laboratory.
Obviously there has been uptake through things like the Oculus Rift, Apple and Google's AR solutions, (cue my kids and their snapchat filters!) and only recently the iPad Pro 2020 released complete with lidar and cameras for enhanced augmented reality. The advance into healthcare is increasing, with Novartis's recent acquisition of a digital health startup Amblyotech to treat amblyopia, "lazy eye".
Novartis acquires digital therapeutic using video games, 3D glasses
To make the most of the advancing availability of these technologies and support innovation across life science and digital healthcare to the benefit of the patient, Pistoia Alliance was delighted to be one of the sponsors of the Virtual Medicine conference's virtual hackathon, co-supporting the Grand Prize. Over 80 people joined the virtual event over a period of four months, built teams and developed immersive technology prototypes for healthcare.
Hackathons by their very nature are collaborative, something close to our heart. It brings people and ideas together and in this case fostered a true learning environment virtually that allowed each and every team to excel. This event showed us in a relatively short space of time key areas where our industry can, and is, using extended reality to the benefit of workers and patients alike in pursuit of effective innovation.
Here's what we learnt:
Hackathon Highlights:
You can see all the submissions on the hackathon page at devpost, please take a look, like and comment on them. Here's some of the highlights and areas the participants tackled:
The Winner:
Mandala Ball - the potential of VR for treatment, recovery and monitoring
Congratulations to Mandala Ball, the stand out choice from strong competition!
Mandala Ball's idea of combining the VR world for a professional looking game, plus using biosensors for monitoring showed an impressive glimpse of the potential of combining VR with scientific diagnostics.
What struck me was how the team put together an impressive proof of concept with the idea of helping children with Attention Deficit Hyperactivity Disorder (ADHD) through a neurofeedback therapeutic VR game. The demo was highly professional but the key area that stood out was the idea of combining the virtual reality headset with a tool like Neurosky's mindwave headset to view and record EEG data in real-time.
Feedback loops like this have the potential to be a highly effective model for for VR/AR applications.
Many Notable entries!
The entrants tackled a number of key themes, here are some of my highlights:
Enhancing Training and Support
There were a number of interesting entries here, they demonstrated that both virtual and augmented reality can really support the training and development of individuals, be that in the hospital environment or the extrapolations such as the scientific laboratory or off-site clinical trials.
Team VRN (Virtual Reality for Registered Nurses)Â showed clearly that virtual reality is a powerful training tool, bearing in mind this was a hackathon, not a product showcase, they developed what looked to my untrained eye a nearly ready to go training simulation:
The team's demonstration takes junior staff through what can be a steep learning curve in often stressful, high pressure environments, giving them the chance to learn in close-to-real examples of critical instrumentation set in high risk situations, yet now risk free!
We were struck both by the concept but also the professionalism of the demonstration, not something usually seen in hackathons, often they are more smoke and mirrors, but not in this case. In these times of social isolation, the ability to learn virtually in a safe and interactive environment should not be overlooked.
Team: AR navigation system for priming ECMO
This is possibly my favourite demonstration in the hackathon, as a creator of hackathons and builder of multiple prototype solutions in my career, this submission to me, is the true ethos of a hack, helping us visualise their idea in a simple and effective prototype. I loved the use of cardboard boxes, plastic bottles and labels to mock up venoarterial (VA) extracorporeal membrane oxygenation (ECMO) equipment for cardiovascular emergency treatment by replacing the function of the heart and lungs.
Their demo clearly shows the true potential of augmented or mixed reality in laboratories, emergency rooms and more. From my experience in laboratories and robotics, this really caught my eye showing how this technology could support scientists, engineers and healthcare professionals in their working environment. Whilst the big tech companies often initially focus their efforts on bringing their XR solutions to the public, for example google glass, the potential user base size was obvious and so they focused on uses for "Joe Public" with ultimately minimal success for glass whilst paying less attention the business use cases - (that's not to say they didn't learn from this and what this tech developed into).
AR ECMO clearly shows the immediate impact potential to many working (and remote working) scenarios at a timely junction as companies such as Apple and Google continue to make the AR toolkits and hardware more readily available to all.
In this instrument training scenario, the interesting challenge will be whether the AR tools can recognise individual pieces in the much more complex set up of actual equipment to move from the icon example above to simply looking at the instrument and the right instruction appear over the right area at the right time and respond to the right action - but that's a challenge to the more technical people than me!
However an industry collaborating together could accelerate the elimination of this problem!
I hope this team follows up on their promise to show a fully working version in the future!
Innovative Rehabilitative Therapy
We've already seen the example in Mandala Ball, a more advanced example with respects to their stage in the product life cycle came from TherapyFlow. The submitter built a fully working game to help the occupational therapist provide a fun and "sticky" rehabilitation programme for the patient.
This was an exciting solution tying gaming to a rehabilitation exercise programme via the Oculus Quest headset with its hand tracking functionality, the submission looked and performed like an off the shelf game.
https://www.youtube.com/watch?time_continue=192&v=AgyqOHEoF6c&feature=emb_logo
Most importantly TherapyFlow has already been accepted into a small trial at the children's hospital in Los Angeles. We wish it every success!
So what did we learn?
In general terms, simply the quality of solutions that teams can deliver with the various extended reality solutions shows the technology is (and has been for a while) ready to impact our industry.
You can see the rest of the submissions, equally as impressive on devpost
The hackathon showed that:
The technology is ready for increased adoption, and the audience are primed! For a long time it has been tried and tested, within medical circles as well as the developer and gaming community and more. The hackathon demonstrated that when we put these differing communities together the sum of the whole can be greater than the sum of the parts. The easier to use tools and hardware definitely underpin this advance.
Its strengths lie in the interaction, all of these will stand or fall by the user experience they deliver - the immersive environment adds new challenges, get this right and it can be a compelling and enjoyable teaching and training tool.
There is strength in the collaborative power of hackathons - once again, collaborating together with a common challenge with passionate and engaged people reaps rewards - used correctly it is a powerful and engaging tool for innovation.
Ideas can overlap and there's opportunities to cross-pollinate the ideas after such events. - there were definitely common themes in the submissions, it is easy to see how the EDVR submitter paired with one of the other teams tackling hand tracking would make a powerful new team, take that further and support him with providing guidance on how to scientifically test and guide his solution would make an interesting proposition.
The challenges ahead in moving from proof of concept to medical testing and validation of impact through outcomes - the regulatory and patient safety implications in healthcare must always be paramount, the industry must support innovative individuals such as these and encourage them to work in our world for the benefit of patients longer term.
What about healthcare and lifescience?
User Experience is key:
The discipline of User Experience is obviously vital with these technologies, their success will stand or fall by how well they engage the wearer / user. Our own UX for Life Sciences project helps drives the adoption of UX in life sciences, the learnings and guides they are building will no doubt help support turning ideas such as these into workable and effective products to the benefit of scientists, healthcare specialists, engineers and maybe most importantly the patient!
Where else can these ideas be adopted?
When you look across all the submissions, themes become apparent, the solutions:
Enabled no-risk, high quality training
Provided real time guidance for technical / medical personnel
Supported patient rehabilitation
Facilitated condition management with monitoring and feedback
These themes easily translate to the life science world as well as the healthcare examples shown. Training and guidance is an obvious opportunity, particularly in laboratories, be that teaching how to use a new instrument, perform aa new procedure or guide the user or engineer in real time. There have been examples of this for a long time now, but the new headsets, phones and tablets will make this evermore easy to deliver.
Guided by strong user focus and design principles this could easily be delivered. A cross industry group would be a great way to accelerate the delivery into laboratories, by supporting the instrument manufacturers, we could deliver common standards and use of training for all.
Equally, building these training solutions for beyond the laboratory could provide better models for testing and patient support beyond the lab or hospital. Perhaps enabling more effective remote clinical trials and the suchlike?
The rehabilitative or condition management scenarios will lend themselves to assistive treatments alongside or instead of medication, the most powerful aspect will be the feedback loop, the concept of "digital biomarkers" being gathered by these technologies will gain further insight into patient management and better understanding of the effectiveness of treatments.
It is easy to see how extended reality solutions can support a more efficient and effective industry many of these scenarios are not areas where it makes sense to go it alone, they are great opportunities for collaboration to better serve life sciences and healthcare. We hope as companies or individuals look into the potential of these technologies they look to collaborate with each other rather than all create their own version of the same problem statement.
I was privileged to be one fo the judges and was impressed with the delivery and quality of the submissions, again my congratulations go out to the teams, it will be interesting to see how and where these ideas end up.
Our congratulations go to Brandon and the rest of the Virtual Medicine team and their sponsors for organising an excellent and stimulating hackathon!
If you have an idea for a VR/AR led project that the industry should collaborate on please register your interest via IP3
Find out more about the Virtual Medicine Hackathon sponsors:
Boston Children's Hospital: http://simpeds.org/
Cedars Sinai: https://www.cedars-sinai.org/
Chariot Kids https://www.stanfordchildrens.org/en/innovation/chariot
Northwestern Mutual: https://www.northwesternmutual.com/
XR Health: https://www.xr.health/
Virtual Medecine: https://www.virtualmedicine.health/
David runs the Pistoia Alliance’s President’s Start-up Challenge and is looking to partner with or sponsor other industry competitions, hackathons and innovation challenges that support start-ups serving the life science and healthcare industries. If your organisation runs a start-up competition or is looking to set one up, please contact David by email: innovate@pistoiaalliance.org