How VirtaMed trains tomorrow’s surgeons in VR

VirtaMed: A Unity case study

Since 2007, VirtaMed has been improving the quality of surgical care through state-of-the-art virtual reality (VR) applications. By leveraging a combination of specialized hardware instruments and data-backed anatomical models, VirtaMed’s VR training environments empower surgeons to master fundamental techniques and pursue continuing professional development in advanced interventions.

Note: This case study contains medical images that may be uncomfortable for some.

The results

The purpose of Virtamed’s robotic simulators is to provide a risk-free, controlled environment for surgeons to practice and perfect their skills. By replicating the robotic systems used in surgeries, they provide a realistic and immersive training experience. The outcomes include:

• A shorter learning curve for robotic surgery compared to traditional methods.
• Unburdened critical resources by abstracting training away from surgical robots needed for patient interventions.
• Reduced or eliminated need for lab training, saving costs and providing surgical teams the opportunity to repeat training.

Read on for a closer look into how the simulations are created and the challenges they address.

The challenges of training surgical teams

Robotics-assisted surgery (RAS) has paved the way for novel procedures that require advanced precision and control, and become necessary for minimally invasive interventions that cannot be performed through traditional methods. As the demand for such surgeries increases, a well-trained and highly qualified surgical team is critical.

Surgical robotics training was traditionally performed with cadavers or medical props on an operating table. However, staff training more commonly occurs with live patients directly, so that training programs don’t impede actual surgeries from being performed.

Surgical simulation provides a risk-free environment for medical personnel to practice and refine their skills. The use of these simulators allows surgeons to practice without blocking the operating room or requiring real patients. Thanks to these benefits, more surgical simulators are arriving alongside a new generation of surgical robots.

The simulator at a glance

Who is it for?

The robotic simulator is primarily used by surgical teams within hospitals, as well as by individual surgeons. It is also utilized in medical schools for training purposes. Additionally, robot manufacturers use the simulator technology to train and educate surgeons about their new robotic solutions.

What does it do?

The robotic simulator serves as a virtual reality platform for surgical training, closely replicating the functions of robotic instrument arms and recreating the surgical space. It is designed to mimic the actual robotic systems used in surgeries, providing a realistic and immersive training experience. The simulator’s 3D software and hardware components are crafted to offer a stand-alone training environment that is nearly identical to the actual surgical console, master controllers, and vision cart.

How widely adopted is it?

The adoption of robotic simulators is quite extensive and growing rapidly. VirtaMed is a leading provider of medical simulation training solutions, and the sole simulator partner of four next-generation surgical robotics manufacturers to date. In addition to existing partnerships producing individualized simulators for a variety of medical devices, VirtaMed is currently developing expanded training capabilities for surgical staff and preparing new hyperrealistic VR modules in Unity to address future advances in robotic- and digitally-assisted surgeries.

Why virtual reality (VR)?

VR in robotic surgical training offers an immersive experience that enhances learning, improves retention, provides a safe practice environment, and allows for accessible and customizable education. Repeatable, low-stress training simulations can reduce or eliminate the need for costly laboratory training. More importantly, virtual training programs improve surgical outcomes to ensure that patients always receive the best care possible.

What hardware does it run on?

Some variations of the simulators use a robotic console integrated in a “backpack”, where the simulator attaches to the robotic console. Other versions use functional replicas of the robotic console designed to be deployed easily by field teams, allowing healthcare practitioners to gain procedure experience conveniently and quickly.

What training feedback tools are included?

VirtaMed’s developers define criteria that are required in order for the learner to be deemed proficient in each simulation. The simulator allows for collection of data such as instrument trajectories during the training session, and provides a feedback report (with a score) for users and trainers.

Unity solutions used

VirtaMed’s team leveraged Unity Industry and a suite of first-party package utilities to simplify their workflow, accelerate development, and build robust simulations. Here’s a brief overview of Unity technologies featured in their tech stack:

Pixyz

Pixyz enables animators and 3D modelers to ingest and render high-fidelity assets and meshes from CAD data in over 70 file types. This pipeline was critical to incorporating data-backed anatomical models and faithfully virtualizing real-world surgical instruments.

Unity Editor

Built-in support for Visual Studio Code and other integrated development environments (IDE) allowed for straightforward development and debugging, while the close integration of modeling tools permitted VirtaMed’s technical animators to confidently manipulate physics-backed world objects in 3D space.

Articulation Body physics

Surgical simulations require the precise rotation of models and robust collision interactions between virtual objects and meshes, especially at specific angles. To accomplish these complex and delicate physics simulations, VirtaMed turned to Unity’s Articulation Body components to create custom physics hierarchies.

High Definition Render Pipeline (HDRP)

Unity’s High Definition Render Pipeline (HDRP) granted modelers an extensive range of parameters to configure realistic visuals and textures. With the flexibility to switch between single-pass instancing and multipass VR rendering, VirtaMed’s engineers could optimize their rendering pipeline for minimal CPU consumption and adapt their simulations to more powerful GPU-equipped devices (such as certain VR peripherals).

Shader Graph

VirtaMed’s engineers didn’t have to worry about writing and compiling custom fragments and vertex shaders. When used in conjunction with HDRP, the Shader Graph utility allowed developers to implement shader node hierarchies with minimal configuration.

Multiplatform deployment

Surgical training simulations rely on low-latency haptic feedback between specialized hardware inputs running in heterogeneous compute environments. Unity’s runtime configuration parameters simplified the process of deploying to different computing platforms and interfacing with specialized medical devices.

Runtime debugging

VirtaMed engineers took advantage of Unity’s built-in performance Profiler to debug behavior and performance issues during testing and at runtime. These tools were essential to fine-tuning both simulation logic and hardware configurations to ensure optimized visual performance and maximum responsivity during live simulations.

Automated testing

With the Unity Test Framework, VirtaMed engineers built tests that could be run in the local development environment and within a simulated output, which helped their cross-disciplinary team adhere to good design patterns and proactively avoid software regression during continuous development.

User reactions

Appreciated by VirtaMed’s customers

Reasons that medical professionals value the fully digitized learning journey enabled by the simulator include:

• The realistic training environment using the latest in graphics technology.
• Integration of real sensorized surgical instruments for hands-on simulation.
• Lifelike, tactile feedback provided via advanced haptic interfaces.
• The versatility and modularity of the simulator platforms, allowing for deployment across many environments including conferences, training centers, and hospitals.

The robotic simulator provides a mixed-reality platform that delivers next-level training for the entire surgical team, directly in the hospital. This reduces or eliminates the need for lab training, which lowers costs and allows for repeated training. VirtaMed’s simulator is remarkably close to the actual robot and provides a seamless experience tailored to each individual’s training needs, making it an invaluable tool for surgeons to hone their skills before entering the operating room.

Looking to the future

Expanding training capabilities

VistaMed plans to continue elevating surgical proficiency by further integrating the robotic simulator platform into the training of surgical teams. This approach allows for training directly in the hospital.

Partnerships

VirtaMed values partnerships and believes that simulation training is an essential component of surgical education and robotic surgery training. Their collaboration with experts and societies aims to train surgeons and medical professionals worldwide to achieve better patient outcomes. Product development and ongoing innovation

VirtaMed is committed to expanding their modules and other products to cater to more procedures, cases, pathologies, variations, and complications.