Case studies

How KLM Royal Dutch Airlines built a VR cockpit trainer with XR technology

Aug 15, 2024|5 Min

Discover how KLM Royal Dutch Airlines’ subsidiary, KLM Cityhopper, is using XR technology to build a VR cockpit trainer

Overview

Creating visually realistic and customized simulation training efficiently using Unity.

Use case

Virtual reality immersive training for flexible pilot training.

Solutions

Unity Engine, Unity Asset Manager

Team members

20 pilots trained monthly and 7 Unity developers.

Meeting the demands of modern aviation

KLM Royal Dutch Airlines and its regional subsidiary, KLM Cityhopper, place a strong emphasis on providing top-notch training for their workforce of over 33,000 and fleet of 110 aircraft. This ensures reliability and safety for the 34.1 million passengers and 621,000 units of cargo they transport annually.

However, managing such a large workforce and extensive asset catalog presents challenges; building and scaling training experiences can be time-consuming and costly.

To tackle these challenges, KLM’s XR Center of Excellence (XRCoE) harnessed Unity’s extended reality (XR) technology to develop an advanced virtual reality (VR) training application: the KLM Cityhopper Virtual Reality Cockpit Trainer. By using Unity, KLM achieved:

Significant reductions in iteration time
Full immersion for trainees using the latest headset technology
Enhanced quality and customization of training scenarios

Read on to explore the development of KLM’s innovative training initiative and discover how it addressed the shortcomings of outdated training methods.

Unity Editor scripting is crucial in creating our training programs. It provides tools to create custom interfaces that our developers use to speed up tasks, allowing them to focus on the details and challenges of creating an extensive training app.

THEO PUIJK / KLM ROYAL DUTCH AIRLINESXR Lead Developer

The challenges of conventional training methods

Outdated training materials: Relying on physical resources and outdated platforms like Integrated Procedures Trainer (IPT) simulators made it difficult to keep training materials current and accessible globally. Frequent manual updates were not only time-consuming but costly.

Poor adaptability: Traditional methods lacked the flexibility to scale and deliver training efficiently based on the varying number of trainees and preferred training platforms (i.e., mobile, desktop, VR). This inflexibility often resulted in scheduling conflicts, underutilized training sessions, or overburdened resources, which could not be adjusted swiftly to meet real-time needs.

The solution: KLM Cityhopper Virtual Reality Cockpit Trainer

KLM’s XRCoE team, made up of 16 people, including 7 certified Unity developers, developed the KLM Cityhopper Virtual Reality Cockpit Trainer for the Embraer E175 and E190 aircraft.

The team leveraged Unity’s 3D development tools to build a realistic VR cockpit, immersing KLM pilots in an environment where they can practice, hone their skills, and enhance situational awareness through functions like:

Interactive buttons
Integrated airplane manuals and procedures
Simulated aviation scenarios

The VR headset literally opens a new world for future Embraer pilots. Using the headset, they can train wherever and whenever they want. Our co-pilots are all young and they might prefer training in the evening or in half-hour stages. This is ideal.

ERNST PONTIER / KLM CITYHOPPERCaptain Embraer and Senior Instructor

Key features of the KLM Cityhopper Virtual Reality Cockpit Trainer

Virtual cockpit: Pilots step into an interactive, computer-generated cockpit environment. They can explore control panels, familiarize themselves with the cockpit layout, and practice ground and inflight procedures.

360-degree point-of-view video: Pilots watch a 360-degree point-of-view video of a flight from the cockpit jump seat. This immersive experience enhances situational awareness and understanding of flight dynamics.

Virtual walkaround: Using a lifelike 3D model, pilots virtually walk around and inspect the aircraft. This helps them understand its physical features and improve overall familiarity.

Image of the KLM Training App in the Unity Editor

Why use Unity to build a training application?

Faster prototyping: KLM’s team used high-fidelity cameras and LiDAR depth scanners to acquire the data needed to create 3D models of their aircraft. The team also collaborated with Embraer, using the aircraft manufacturer's 3D modeler, to manually create models of the E175 and E190 aircrafts, as comprehensive CAD drawings were not available.

Unity’s technology facilitated quick prototyping and visualization, which was essential for KLM’s iterative development process.

The Unity Editor provided an intuitive platform that allowed the team to quickly develop and refine various design concepts and training scenarios. KLM created all 3D assets from scratch for their VR cockpit trainer and Unity’s tools offered full control over rendering and interactivity, ensuring realistic representation of the physical cockpit.

Smooth performance and rapid iteration: Unity’s multi-platform support facilitated smooth deployment to the Meta Quest 3, while its integrated development environment significantly accelerated the development iteration process. The ability to test and visualize features in real-time within the Unity Editor, without the need for frequent builds and deployments, dramatically reduced deployment lead times.

This approach has been particularly valuable for decision-making and refining complex features, as it enables on-the-spot adjustments and immediate testing. By minimizing the full build and deploy cycles, the team at KLM could allocate more time to improving the overall quality and effectiveness of the training application.

Iterating and testing an application is easy using Unity. We can simply sit next to a customer, go through the application, and edit anything on the fly. This reduces the time we spend waiting for feedback and adjusting.

THEO PUIJK / KLM ROYAL DUTCH AIRLINESXR Lead Developer

Improved visual asset quality and customization: KLM worked with Embraer to develop various features for the training application, like hand-tracking and a custom procedure node system design specifically for KLM’s developers to easily create new procedures.

The team also leveraged Unity’s support for custom shaders and materials. This significantly contributed to elevating the application’s quality and enriching the visual aspect of the training experience.

Streamlined asset management: Throughout their journey of building the KLM Cityhopper Virtual Reality Cockpit Trainer, KLM tested the Unity Asset Manager to manage assets within the Unity Editor. The company is still testing to ensure that the Unity Asset Manager can be integrated within their IT infrastructure, but they did see the solution as a major advantage for streamlining asset management.

The potential for efficient distribution, storage, and updates is appealing. The ability to seamlessly manage assets within the Unity Editor could significantly enhance our workflow, making it an area of interest for future implementation.

WERNER SOETEMAN / AIR FRANCE-KLMSenior IT Manager for Fasttrack & Innovation, Head of XR Center of Excellence

Expanding the VR training experience

Looking ahead, KLM plans to further enhance their VR training application by introducing a multiplayer mode, allowing pilots to train together in real-time. This innovation will continue to drive their business growth and solidify their position as a leader in aviation training.

Unity is the backbone of our development process. Not only do we create our training programs with it, but we also use it to create extensive tooling that helps us with maintaining and developing those trainings.