Adaptive AI meets XR: How Lucid Reality Labs and Unity redefine hands-on medical training

Empowering technology-enabled education that improves patient outcomes: A Unity case study

Combining adaptive AI mentoring and immersive simulation to help practice high-risk procedures.

Lucid Reality Labs is a leader in extended reality (XR) and AI enterprise solutions, and a long-time Unity partner-specializing in digital twins, immersive training and simulation, performance advancement, spatial product activation, and agentic AI.

The company, part of the Unity Partner program, creates high-fidelity immersive content that improves learning, expedites processes, and facilitates data-driven decision-making in high-stakes sectors like robotics, healthcare, manufacturing, and education. Its mission is to reduce risk, boost workforce proficiency, and optimize complex operations while providing clients with quantifiable growth, increased efficiency, and improved precision - all while making a significant, human-centered impact.

Lucid Reality Labs works with leading organizations to address real operational challenges through intelligent, immersive XR solutions built on Unity. Using a business-first strategy, the company connects the workflows and KPIs of each client to Unity's capabilities, producing scalable, platform-agnostic apps that enable teams to practise, learn, and work anywhere.

Proof in practice: Impact at a glance

Combining the precision of an immersive environment with adaptive AI guidance better prepares anesthesiologists for real-world challenges. This approach has demonstrated measurable benefits in building clinical confidence and procedural readiness.

Key results from the Lucid Reality Labs solution include:

• 437 participants from 18 countries completed 491 intubation scenarios.
• 71% of clinicians rated the simulator useful for teaching, and 65% expressed interest in adopting it for continuous training.
• Participants reported enhanced procedural understanding, stronger muscle memory, and faster performance in subsequent sessions.
• Real-time adaptive feedback enabled quicker error correction and shorter learning curves, improving skill retention.
• The system’s scalable, repeatable framework ensures consistent, high-quality training experiences across institutions and regions.

Their XR intubation simulation solution with integrated AI medical agent developed for Medtronic, and powered by Unity, was recognized by the Auggie Awards, Webby Awards, and Telly Awards for innovation in AI and immersive technology. This success reinforces the value of immersive, AI-supported education in advancing medical skill development and improving patient outcomes.

An international study published in the British Journal of Anaesthesia validated the impact of this immersive, AI-supported training, showing significant improvements in procedural understanding, muscle memory, and overall performance.

As further recognition of its impact, this solution was recently featured by Samsung as a transformative healthcare solution during the Galaxy XR launch event in Oct, 2025. Showcased live by Prof. Dr. Patrick Schoettker, it is a compelling proof point of how Unity’s real-time platform enables enterprise-scale XR training across the next generation of Android XR devices.

The urgent need for scalable, data-driven medical training

The world will face a shortage of 10 million health workers by 2030. AI and XR technologies can help bridge this gap by closing training disparities, expanding access to expert guidance, enhancing patient safety, and scaling clinical skills development.

Anesthesiology training demands precision, adaptability, and flawless execution. Yet traditional models face persistent barriers of limited access to expert instructors, reliance on physical simulation labs, and difficulty exposing learners to complex or rare patient scenarios. These limitations hinder scalability, reduce consistency, and impact readiness for real-world applications.

Research shows that even experienced clinicians require continuous, hands-on practice to maintain confidence and refine decision-making across diverse patient profiles and equipment setups. This shows the need for a flexible, data-driven training system that enables scalable, high-fidelity simulations and can provide personalized feedback in a safe, controlled environment.

Transforming intubation training with adaptive AI and extended reality

AI-empowered immersive intubation training, built by Lucid Reality Labs for Medtronic, and powered by Unity, was developed in collaboration with Lausanne University Hospital (CHUV) and Prof. Dr. Patrick Schoettker, Head of Anesthesiology. This medical training redefines expert-led education through adaptive AI mentoring and immersive XR simulation.

The experience recreates a high-fidelity operating room-of-the-future which enables clinicians to safely practice complex intubation procedures under the guidance of an AI-powered digital twin of Dr. Schoettker. The AI mentor provides real-time feedback, clinical rationale, and step-by-step guidance aligned with medical protocols.

Learners progress through various patient profiles and difficulty levels. The system measures precision, timing, and decision-making, providing feedback and suggestions for individual skill development.

Designed as a risk-free, hands-on learning environment, it allows clinicians to train anytime and anywhere-at hospitals, universities, or remote learning sites while ensuring continuous updates to reflect the medical standards and relevant expert knowledge.

The training system uses Unity’s XR Framework and Interaction Toolkit for cross-platform consistency, modularity, and high-fidelity real-time simulation. Building on this foundation, Lucid Reality Labs’ proprietary iXR™ platform works in synergy with Unity, integrating multi-role collaboration, analytics, and secure content management within a unified ecosystem that combines clinical precision with enterprise scalability.

Made with Unity: Technical approach and build

Unity is at the simulation's core, enhanced by the Unity XR Framework, XR Interaction Toolkit, and Lucid Reality Labs' proprietary iXR™ platform. This combination creates a flexible foundation for modular design, multi-role simulation, robust analytics, and seamless content delivery. The iXR™ platform introduces data-driven insights, adaptive learning scenarios, and precise performance tracking essential for medical-grade simulation, extending Unity’s capabilities.

• DevOps built for scale and consistency: A secure continuous integration / continuous deployment (CI/CD) pipeline formed the backbone of scalability and reliability. Development and QA teams worked within a unified DevOps environment featuring automated testing, version control, and streamlined deployment. Each build was validated across supported XR devices with minimal downtime, enabling rapid iteration, reducing manual intervention, and ensuring stable, version-controlled updates for high-stakes clinical training.
• Device-agnostic architecture: Developed in Unity using the OpenXR Plugin, the simulation is hardware-agnostic by design. Whether running on hand-tracking, controller-based headsets, or eye-tracking devices, the experience remains consistent and intuitive - allowing deployment across hospitals, universities, and training centers worldwide regardless of XR hardware.
• Performance, reliability, and realism: To meet the precision demands of clinical training, the team implemented adaptive rendering, latency optimization, and deep performance profiling. The result is smooth, clinic-grade frame rates with responsive instrument interaction, supported by telemetry from the iXR™ platform for continuous refinement and real-time feedback loops.

Why Unity

Lucid Reality Labs selected Unity for its proven real-time performance, cross-platform scalability, and the industry’s most comprehensive XR development ecosystem.

Core Unity tech powering the solution includes:

• Unity XR management with OpenXR Plugin - Delivers consistent cross-device performance through a unified interface for multi-hardware deployment.
• XR Interaction Toolkit - Provides intuitive, physics-accurate interaction with instruments and patients, essential for procedural realism.
• Universal Render Pipeline (URP) - Balances high-fidelity visual output with optimized performance for standalone headsets.
• Unity Batch Mode for CI/CD - Automates testing and build pipelines, improving delivery speed and application stability.
• Performance Profiling Tools - Enable deep optimization, ensuring clinical-grade frame rates and sustained reliability.

Design-to-deployment process: From prototype to clinical-grade XR training

1. Clinician-centered design: Built for clinicians and with clinicians, the simulation was shaped around authentic medical workflows and training needs. Each design stage focused on realism, mirroring procedural flow, safety standards, and clinical decision points to align success metrics with real operating room outcomes.

2. Prototype and test: Early Unity prototypes were created to validate interaction models, AI mentor logic, and UX flow. Iterative testing with anesthesiologists and clinical educators provided continuous feedback, refining realism, usability, and instructional accuracy to meet both educational and operational expectations.

3. Content & analytics integration: Procedural scenarios, instruments, and performance indicators were developed in parallel with learning objectives. Analytics pipelines were embedded early using iXR™, enabling real-time data capture, measurable progress tracking, and continuous improvement across training sessions.

4. Optimize and pilot: The experience was performance-tuned across XR devices and stress-tested with subject matter experts to validate accuracy, stability, and clinical authenticity. Telemetry insights from pilot sessions informed final refinements, ensuring consistency, reliability, and readiness for enterprise deployment.

Key capabilities of the XR healthcare simulation

The experience combines immersive realism, adaptive AI mentoring, and clinical-grade telemetry to scale expert training safely and consistently.

• Hyper-real operating room simulation: Replicates real clinical pace, setup, and procedural pressure with dynamic difficulty adjustment to build confidence and reduce cognitive load.
• AI mentor (Digital Twin): An intelligent digital twin of a leading anesthesiologist provides real-time guidance, clinical reasoning, and adaptive feedback aligned to validated medical protocols.
• Clinician-validated design: Developed with anesthesiology specialists to mirror real equipment, workflows, and safety standards for direct real-world skill transfer.
• Performance analytics and objective assessment: Measures timing, sequence accuracy, instrument handling, and decision-making for structured debriefs and continuous skill development.
• Data integrity and secure governance: The iXR™ platform manages content, user roles, and analytics with full traceability, version control, and compliant data handling.
• Evidence-based outcomes: Every session logs interactions and results, enabling measurable insight into skill progression, training quality, and learning retention.

Scaling from simulation to adaptive AI training

From prototype to global training standard, the project progressed through three major innovation phases, each expanding the impact of Unity-powered medical education:

Stage 1 - VR foundation

A Unity-built immersive training module for airway management, later validated in an international British Journal of Anaesthesia study with hundreds of clinicians across 18 countries.

Result: XR simulation proved clinically realistic, repeatable, and scalable for high-stakes practice.

Stage 2 - AI medical agent

Introduction of an intelligent AI mentor delivering real-time feedback and reasoning inside the same Unity environment.

Result: Simulation evolved into interactive, guided learning with performance-aware instruction and adaptive feedback.

Stage 3 - Immersive interface for intelligence

XR becomes the interface for adaptive AI, visualizing knowledge, responding to learner decisions, and improving through telemetry. Enhanced hand tracking and deeper analytics through iXR™ strengthen personalization and skill development.

Looking ahead: Expanding adaptive AI and immersive learning

Lucid Reality Labs continues to advance its AI-driven simulation framework to support enterprise-level medical training and high-stakes procedures that demand precision, adaptability, and measurable performance. With Unity as the core real-time engine where intelligence meets interactivity, the next phase focuses on strengthening the synergy between AI and XR, enabling training that learns, evolves, and generates new scenarios on demand.

Future developments include:

• AI-generated scenario creation powered by the iXR™ platform, enabling on-demand training modules that adapt to learner needs and reduce manual development effort.
• Smart content management with Unity Asset Manager, ensuring centralized updates, traceability, and seamless deployment across organizations and devices.
• Universal accessibility, delivering consistent adaptive training across XR headsets, desktop, and tablet, expanding reach to every learner regardless of hardware.
• Continuous optimization loops, where analytics feed improvement and skill insights back into the system to refine decision-making and accelerate learning curves.
• Enterprise-grade compliance, with expert-validated modules maintaining alignment with GDPR, HIPAA, ISO 27001-based standards, and evolving clinical best practices.

By combining Unity’s real-time ecosystem with Lucid Reality Labs’ adaptive AI and iXR™ platform, this solution continues to evolve into a self-improving, scalable training system - designed to elevate clinical readiness, enhance patient safety, and shape the future of healthcare education.