Coco VR

by Pixar & Magnopus
The studio
Magicians of VR spectacle

Magnopus founders Ben Grossmann, Alex Henning, and Rodrigo Teixeira have credits in some of the most effects-heavy film productions in the last decade, including the Oscar-winning Hugo, Star Trek: Into Darkness, The Amazing Spiderman, Alice in Wonderland, and The Jungle Book. For well over a decade, the ultra-talented team at Magnopus has been bending creative technology to their wills to create impressive visual feasts.

“We started Magnopus because films were hitting a ceiling and we were running out of impossible spectacles to wow people with. We saw directors struggling to engage audiences, and audiences looking for more than a theater or TV could provide,” said Ben Grossmann, CEO. “We knew we needed to put audiences inside those experiences, but that we’d never be able to do it within the existing system, so we left to start fresh. Now we’re building experiences and technology that allow people to become an active part of the story, whether it’s a mixture of the physical and digital worlds or a completely imagined world.”

They delivered some of 2017’s most impressive VR experiences in Mission:ISS and the Emmy-nominated Blade Runner 2049: Memory Lab, gaining valuable expertise in the process. “Developing for VR is fundamentally different from creating VFX for traditional films or video games,” explained Lap Luu, CTO. “Making amazing VR content is not the same as making films, TV or games. It’s like all three at the same time.”

(Top) Magnopus co-founders Rodrigo Teixeira, Alex Henning, and Ben Grossmann. (Bottom) Pixar’s John Halstead, Supervising Technical Director

The project
Inviting the living to the Land of the Dead

In Coco VR, Magnopus and Pixar combined their creative talent and technical wizardry to transport players into Pixar’s beautiful vision of the Land of the Dead as seen in their animated film Coco. “We set out to both explore the creative opportunities of the medium as well as push the technology to its limit,” said John Halstead, Pixar’s Supervising Technical Director.

“So instead of just doing a narrative short of the film, we designed an active experience that gives users the opportunity to visit many of the beautiful locations in the film, including taking a gondola ride and enjoying gorgeous views of the surrounding city.”

As well as experiencing the world hundreds of feet above the city – courtesy of an Oculus Rift headset – players experience the vibrantly colored Land of the Dead as they wander through a series of mini-experiences that include a costume shop, an art gallery, a cinema, a photo booth, and a musical performance.

Coco VR – like Magnopus’ Blade Runner 2049: Memory Lab – is nominated for an Emmy this year in the Outstanding Original Interactive Program category. Players are encouraged to experience Coco VR on their own or with up to three friends through each of the activities. “The multiplayer experience is where Coco VR really comes to life,” said Alex Henning. “It was built to be a social experience from the ground up and there’s no doubt that visiting the Land of the Dead with friends elevates the fun.”

Watch Coco VR's trailer
The reveal
The challenge of Pixar standards in VR

The multiplayer functionality of Coco VR was among the first features Magnopus implemented. While important to the eventual enjoyment of the final experience for players, multiplayer first proved invaluable in the highly collaborative design and iteration process with clients at Disney and Pixar.

“One of the most amazing things about working in VR with a client is how effective communication can be,” observed Henning. “We’re able to discuss complex design topics while freely moving around, pointing to specific areas like we’re walking around a set on a movie: ‘Change this detail, make this alteration, let’s move this over here, what if we put this stuff there and had players experience things this way?’ All of these suggestions and discussions become infinitely more productive, shortening iteration time on projects.”

Multiplayer isn’t just for fun anymore

Animation worthy of a hero

On entering and exiting the experience, players are treated to short conversations with Miguel, the hero of the films, and they can also interact with Hector during an enjoyable performance sequence. Pixar’s focus on character meant these heroes, so recognizable from the film, needed to translate 1:1 to a new medium.

Pixar’s storytelling style is heavily dependent on their advanced facial animation, so while Magnopus optimized nearly every model from the film, the hero character faces and animations are rendered in the exact format of the original. Normally in a VR experience, the textures are the largest files, but in the case of Coco VR, the animation files were massive and created with Pixar’s proprietary animation system, prompting Magnopus to craft custom tools capable of accommodating the files in real time in Unity.

Hector in the Editor

Hector in the Editor

In addition to the facial animation, the movement of the skeleton bodies is extremely important to the immersion of the experience. The team utilized Final IK, an inverse kinematics solution available on the Asset Store, to implement the skeleton body animation of players and their friends.

Massive detail, minimal cost

Of the many successful aspects of the Coco VR experience, one of the most impressive is the visualization of the Land of the Dead. “Pixar created one of the most detailed, extravagant, colorful, brilliant visualizations for the Land of the Dead for the Coco film,” Luu reflected. “Everyone working on this project felt a need to bring that vision to life in VR as closely as possible. We wanted players to really feel like they’ve crossed to the other side, just like Miguel did in the movie.”

“That was a key part of the project for us,” said Pixar’s Halstead. “Our team spent a lot of effort redesigning elements from the film to work natively in this fresh format. And part of the challenge was to make sure they worked smoothly both in the Unity engine and in the experience itself. To get it right, we went through a number of steps before handing off the animations to Magnopus.”

Incredible size and scale made affordable for VR.

Incredible size and scale made affordable for VR.

In order to achieve the level of detail they hoped for, Magnopus cultivated new techniques that blended a variety of 360-degree projections as the environment, allowing them to use actual RenderMan renders from the film in some cases. “What you’re actually seeing is a multi-layered experience of true 3D assets up closer and more projections the farther the scenery is in the distance,” said lead 3D artist Luke Schloemer. “You can think of it as skybox 1.5.”

The detailed layering of environments is used everywhere in the experience but is most apparent in an eye-popping gondola ride through the city. It’s a mixture that easily tricks the brain into seeing a cohesive whole and allows for a stunning amount of detail in a VR environment.

“It was really interesting seeing the finished movie for the first time,” recalled Henning. “Having been in the VR experience first and having stood in the middle of the square and ridden the gondola, the experience of seeing the movie was almost like looking at vacation photos from somewhere I had already visited.”

Learn more about how you can build incredible interactive experiences in VR and AR with Unity!

We use cookies to ensure that we give you the best experience on our website. Click here for more information.