You’re part of the VR story in Bonfire with AI-based characters

How award-winning Baobab tapped Unity to create immersive experiences for mobile

Baobab Studios: A Unity case study

Bonfire is a cinematic virtual reality (VR) movie that enables viewers to become actual participants. You communicate with aliens and determine how the story unfolds and, ultimately, how it ends. It features comedian, writer and actress Ali Wong (star of Netflix’s Always Be My Maybe), and is six-time Emmy-winning Baobab Studios’ most-recent project, following fan favorites like ASTEROIDS! and INVASION! 

 

The Goal

To create an interactive VR movie featuring truly meaningful character relationships.

Platforms

Oculus Quest, Oculus Rift, PlayStation VR, and HTC Vive

Project staff

20

Location

San Francisco, USA

Once upon a time ... on an alien planet

In Bonfire, you are faced with the challenge of surviving on an alien planet with only a bonfire and Debbie, a robot sidekick (voiced by Ali Wong). You encounter Pork Bun, an alien creature that cannot speak our language. Pork Bun communicates solely through non-verbal, animated performances, which are procedurally generated based on how you interact with it. The result is a completely non-linear narrative, where all the character responses are driven by complex artificial intelligence and emotion systems that respond to your behaviors – all rendered in real-time.

The results:

  • Created an award-winning, interactive VR movie showcased globally 
  • Built a real-time rendered, highly stylized and cinematic alien world 
  • Extended VR storytelling by empowering audiences to be part of the story
  • Enabled people to experience Bonfire anywhere via mobile VR headsets
Gather around the VR bonfire

Making viewers part of the story

Baobab’s guiding vision is to create immersive storytelling experiences where viewers take an active role in the narrative. “We believe that storytelling’s greatest strength is the ability to create emotional experiences through vivid characterization. If we can make you truly care about the characters, you’re more invested in the outcome and you might even become a more compassionate person,” says Baobab co-founder and CTO Larry Cutler. 

Validated with industry awards and positive reviews, Unity and Baobab pushed technological limits to achieve this aim. This critically acclaimed VR experience puts you in a difficult and emotional position: Will you protect life on a strange planet or will you choose this as a new home for humanity? 

Real-time rendering for mobile VR

Baobab’s previous projects were designed to run on headsets such as the Oculus Rift and PlayStation VR that are powered by high-end hardware. For Bonfire, they targeted standalone VR headsets like the Oculus Quest, where users are completely mobile, untethered to a computer. Bonfire was initially built, optimized, and released as part of the Oculus Quest launch – one of just a few non-games to get the nod from Oculus.

But rendering a VR experience in real-time posed a challenge because mobile chipsets have approximately an order of magnitude less compute power compared to high-end headsets. 

To the rescue came Baobab’s art director, Oscar- and Emmy-winning Kal Athannassov, who developed a stylized design that gravitated towards simple graphic shapes that work well on mobile. Over the course of production, the team tackled numerous optimization challenges such as rendering organic characters like Pork Bun in full fidelity and creating a stylized lighting aesthetic for a campfire that casts shadows on all the dynamic characters and environment objects.

Baobab used Unity’s Scriptable Render Pipeline (SRP) to produce this graphically rich environment. According to Nathaniel Dirksen, Bonfire’s digital supervisor, “The new Scriptable Render Pipeline was essential for our ability to render Bonfire in real-time on mobile VR headsets like the Oculus Quest. With the modular architecture, we could include only the rendering functionality we specifically required for our desired look. For example, we created a custom lighting technique geared for casting shadows from our flickering campfire. SRP let us maximize the rendering resources we needed to bring our cinematic alien world to life.” 

Bonfire by Baobab Studios

AI and non-linear storytelling

The team created sophisticated character AI and non-linear storytelling systems in Unity, enabling the viewer to take on the role of a main character. They made significant creative and technical advances in artificial intelligence and emotions that allowed them to evoke empathy, revealing not just a character’s nature but a viewer’s nature, which builds upon the great traditions of storytelling. 

For Pork Bun’s AI brain, Baobab aimed to generate procedural animation that has the same quality character performance as if it had been hand-crafted by an animator. In short, Pork Bun’s personality needed to be embedded in the AI system. For a long time, the AI system produced plausible motion but it was mechanical and did not reflect Pork Bun’s true character. Unity’s flexible animation framework was critical for enabling them to capture the “illusion of life” as Pork Bun reacts to you.

Inspired by stand-up comedy

Despite the serious themes of trust and survival, Bonfire is a comical and casual experience with a much longer script than people might think. The team designed a dynamic system modeled after Ali Wong’s stand-up routines, which are made of bits, jokes and chunks, linking everything together. Ali recorded all of these variant lines to ensure her character would respond appropriately. 

Dirksen explains, “This framework extended well past how we thought about the script – the entire technical approach is built on our proprietary Storyteller system, designed to code for a typical comedy set. These building blocks could be easily represented using Unity’s Timeline or Animator Controllers as appropriate. By seamlessly transitioning between them, based on the user’s actions, Debbie and Pork Bun can deliver responsive performances, while ensuring their actions still feed into a compelling narrative.” Debbie procedurally reacts to the viewer’s actions – with different animations and voiceover lines – delivering an improvised comedic performance that’s unique for each viewer.

Characters put you under pressure

Taking advantage of Unity’s Animation toolset, Bonfire delivers high-quality, animated character performances in VR. But it’s not just about the quality of the graphics and animation, it’s how Unity enabled Baobab to bring the world and characters to life by putting pressure on the viewer to make meaningful and difficult decisions – making it feel real.

As Cutler says, “In Bonfire, your relationships with the characters change throughout the story as we raise the stakes. In VR, we can enable you to act on the empathy you are feeling and actually do something compassionate. For Bonfire, Unity enabled us to create an immersive VR world where characters react and respond to what you are doing, unlike our past experience in feature animation.”

Equipped with Unity’s latest VR tools and multiplatform support, Baobab is connecting with audiences like never before. By creating unique storytelling experiences they are breaking down physical barriers and transporting audiences – wherever they are – into immersive new worlds. 

In their words

“We believe that storytelling’s greatest strength is the ability to create emotional experiences through vivid characterization. If we can make you truly care about the characters, you’re more invested in the outcome and maybe even become a more compassionate person.”

Larry Cutler, Co-Founder and CTO, Baobab Studios

“The new Scriptable Render Pipeline (SRP) was essential for our ability to render Bonfire in real-time on mobile VR headsets like the Oculus Quest. SRP let us maximize the rendering resources we needed to bring our cinematic alien world to life.”

Nathaniel Dirksen, Digital Supervisor, Baobab Studios

“For Bonfire, Unity enabled us to create an immersive VR world where characters react and respond to what you are doing, unlike our past experience in feature animation.”

Larry Cutler, Co-Founder and CTO, Baobab Studios

We use cookies to ensure that we give you the best experience on our website. Visit our cookie policy page for more information.

Got it