How Kluge Interactive punched up VR gesture recognition in FINAL FURY

Kluge Interactive has been developing immersive games since 2008. Their flagship title, Synth Riders, a VR rhythm game, shipped in 2018 and the team has spent the past seven years delivering a steady stream of music packs. These entertaining content updates have elevated the IP into the stratosphere of A-list musicians and nationwide sports events.

The team has no plans to slow down. They combined their VR experience and passion for fighting games to create FINAL FURY. The multiplayer title mixes classic fighting game mechanics with actual fighting. Players make every punch count, either by sparring solo or challenging the galaxy’s best.

We talked to Rob Alvarez, producer of FINAL FURY, and David Montesdeoca, the game’s lead programmer, to discuss how the team harmonizes immersion and speed to create a next-level VR fighting game.

How would you describe FINAL FURY?

Rob Alvarez: The concept behind FINAL FURY was born from a deep love for classic arcade fighting games, reimagined through the lens of modern VR technology. From the very beginning, we kept asking ourselves one core question: “How can we make every action, from throwing punches to dodging attacks, feel as real and visceral as possible?”

Turning that vision into a fully playable experience came with its share of challenges, especially in striking the right balance between the immersive nature of VR and the intense, fast-paced combat that defines the genre. Early on, it wasn't easy to adapt those mechanics to VR, but through constant iteration, we eventually found the sweet spot.

How does the game’s unique camera system increase gameplay immersiveness?

David Montesdeoca: FINAL FURY uses a modular multi-camera system designed specifically to support immersion across different gameplay states, perspectives, and player experiences.

We support multiple camera types: first-person, third-person, spectator, 2D, and an “Out of Body” camera used during cinematic moments like grabs or situations in which the player does not have control. Rather than treating these as isolated systems, we’ve built a unified Camera Service that manages them as either a single logical entity or as separate cameras when specific effects are needed.

Here’s how it works:

• Each camera is driven by a shared event system, but can choose whether and how to respond to events. For example, when a grab is triggered, both the first-person and “Out of Body” cameras may activate effects, but in different ways.
• The system is data-driven and highly scalable. We can add new camera behavior without impacting other systems or needing special logic.
• We split the world into three main visual layers: HMD, 3rdPerson, and HMDAnd3rd. This lets us control visibility by shifting object layers at runtime and allows some elements to appear only in first-person views while being hidden from third-person or cinematic perspectives.

With this setup, we can balance player agency in VR (throwing punches and doing gestures and powers from a first-person view) with more cinematic payoff moments that are more satisfying when viewed externally.

Why was perfecting the combat gestures so important?

Rob: Combat gestures are the core of the immersive fighting experience that we want to deliver. While FINAL FURY is a love letter to everything we love about classic arcade fighting games, it’s also a challenge to make players sweat it out and feel like they are in the arena doing the fighting.

How did the team implement the gesture-based control system?

David: Gesture recognition has been one of the most technically and creatively challenging aspects of FINAL FURY’s design. Our goal has always been to ensure players feel like the fighter, not just someone controlling one.

The key pieces of our system are:

• Input tracking: At the core, we built a lightweight system that continuously tracks each hand's position and orientation relative to a central reference point. It’s usually located at the player’s head (center of mass). This helps us normalize inputs, minimizing head movement or body orientation as variables that could affect gesture detection accuracy.
• Gesture utilities: We use a set of low-level math utilities to detect patterns like “Is the hand moving forward?” or “Is the hand on the left side of a detection zone?”
• ActionTrigger system: We define complex gestures through what we call ActionTriggers. Each ActionTrigger listens for specific gesture sequences using our utilities.
  • The detection process includes a state system with three possible states:
    • Failed: Input does not meet requirements
    • Partial: Gesture appears to be starting correctly
    • Complete: Gesture has been successfully executed

The Partial state is very important as it’s what helps us prioritize gestures based on context, provide real-time feedback to the player so they know they’re on the right track, and telegraph the intention to opponents.

Here’s an example of how our “Grab Gesture” works:

1. The system checks that both hands are far apart and positioned symmetrically on either side of the center of mass. This triggers the Partial state (the “start of the grab”)
2. If the player brings their hands quickly together, the gesture is Completed and the action fires
3. If the movement is too slow or inconsistent, the system enters the Failed state and the gesture is canceled

This approach has allowed us to combine fluid input with precise actions, and we continue refining it based on community feedback.

How did Unity come into the mix?

Rob: Honestly, it wasn’t even a question on our part. Our teams have been using Unity for years for Synth Riders and it’s been a great experience. It gives us the ability to seamlessly work across platforms.

The Engine empowers us to bring together a wide range of tools and systems to create something dynamic and responsive as we build FINAL FURY. As a result, we get to iterate pretty quickly, which is exciting for us. It provides us with the flexibility to test new game concepts with minimal stress.

Unity's plug-in ecosystem has been a big advantage. Tools like NodeCanvas have been invaluable for building complex AI behaviors in a visual and modular way. We’re also using Unity’s OpenXR support and the integration has been surprisingly smooth. It’s practically plug and play, which has made VR development far less painful than expected.

Unity’s built-in support for PlasticSCM has helped us strike the right balance between simplicity and the power we need to manage our project’s codebase.

The team brought pro gamer Justin Wong on as a consultant for the game. What is his involvement?

Rob: We very publicly discussed our need to change approaches as the initial development of FINAL FURY shifted. We delve further into that in our first FINAL FURY blog. As we got deeper into the process, we wanted to make sure that we were not only on the right track, but able to dial things in even tighter. Beyond putting the player in the fighter’s shoes, we wanted to tap into what makes a good competitive fighting game. So, it only made sense to approach Justin for his input.

In multiple play sessions, we’d get into the game with Justin, show our designs, and have some matches. His feedback about balancing and fighting mechanics has been really valuable. It usually covered areas that only an expert fighting game player would bring up. In fact, he’s helped us see things we would not have otherwise, and his involvement has made some of the game’s mechanics feel a whole lot better. Also, I must confess, he did beat us at our own game.

What’s next for FINAL FURY?

Rob: We’ve got big plans for what we’re looking to add to the game during Early Access. The FINAL FURY team continues to explore new ideas and improvements, pushing the gameplay further with every update. New fighters, new arenas, new moves added for all fighters – that’s just the start. Some of it is already being built, other features are actually going to be up to the players.

Our dev team plans to join the fights, listen to what the players are saying, and continue to improve the game. The “release→play→listen→iterate” method is how we shipped Synth Riders back in 2018 and it worked out really well for us.

Our ambition is to become the go-to reference for arcade-style fighting games in virtual reality, and with the dedication of our team, we’re confident we can make that a reality.

To read more about projects made with Unity, visit Unity’s Resources page.