James Liu, Head of Infotainment Platform and Experiences at Mercedes-Benz, discusses how the company is embracing Unity’s technology to build innovative in-car experiences for drivers and passengers.
Transcript
My name is James Liu, and I'm the Head of Infotainment Platform and Experiences at Mercedes-Benz. In essence, I'm responsible for anything the customer sees inside the car. I think the main thing is really this giant digital canvas that we've given to our customers. In our newest GLC, we give customers a 39-inch pillar-to-pillar hyperscreen. And on those screens, we can provide immersive experiences. And this is really where Unity comes into play. We want to make those experiences exciting and immersive. Unity really provides that 3D immersive, not just end-user experience, but a pipeline and a toolkit that allows us to bring that to life. For example, we have the 3D instrument cluster, we have 3D maps, we even have really cute 3D avatars that spring to life when you ask, “Hey, Mercedes”. Customers can select three different avatars from a classic Mercedes-Benz star, to a little cute robot that actually dances around and changes color when you're making jokes at it. The customer reaction, they are blown away.
Calling Unity just a game engine, I think limits its real use potential. We have used Unity not only for pre-visualization and creating ideas, but creating safety-critical instrument clusters in the car. I think a lot of times when people think about screens in the cars, they worry that, hey, is it gonna be distracting? One of our most important pillars of the entire company's ethos is safety.
One thing, we actually have a lot of animations in the vehicle, including a camera that detects exactly where you're looking. So if you're a customer and you're driving, if you are looking at the screen too much, or you look at your passenger side, we actually digitally fade out the content to not distract you. On top of that, every single image in there, we actually apply AI algorithms to make sure that we are not creating too many details in the wrong places.
How has Unity helped us be more creative? The fact is, we are as sharp as our tools. In the old days, you're on a whiteboard sketching something out. You hand it off to a designer, designer sketches something, hands it off to maybe a 3D supplier, and then you wait. But now with these tools, that loop between ideas to experience gets shorter and shorter and really allows us to iterate and therefore be more creative.
My most surprising discovery while using Unity? Really it's actually about creating a team from a classic automotive HMI perspective. You know, we don't really have 3D artists, technical artists, or production artists. So a lot of our work was actually training our own teams. Once you have that talent, the real creativity starts. And that thinking between designing 2D images versus thinking in 3D and creating that whole experience, that was really amazing.
Performance is such a critical part of a Mercedes-Benz experience. When a customer is driving, we want their attention fully focused on the road. So anything that is distracting, from a screen stutter to the fact that if I wanna touch a button on the screen and it's not as responsive, those all lead to these cognitive seconds of distraction. And we know that responsiveness and performance is such a big part of lowering driving distraction.
Why did we choose Unity? Well, we actually, like all good automotive companies, we created a bake-off. And at the end of the day, well, Unity won because of three major reasons. One, performance. Two, scalability. The ability for us to work with our partnership, of course, with a whole world of developers. And the third one is actually service. We had such a great experience working with the Unity team that we felt that this partnership was something worthwhile for us to invest in. The team loves using Unity. The joy of working with such an industry standard tool, I think really helps our team grow.
This is just the start of a journey for us. The next big step in automotive technology is really AI. We're seeing quite a lot of vehicles, ourselves also, enabling what's called level three driving, which is really eyes off the road driving. We're creating these more immersive experiences, AI driven, gen AI driven experiences, where customers can really interact with the car as if it's a robot. So a customer can sit in our vehicle, eyes off the road and have a natural conversation with the vehicle. The experience is really like talking to a friend.
We believe that the future of vehicle experiences are not only static but also stationary. We're giving customers back time, and we would love to fill that time with really wonderful immersive entertainment experiences. Comfort experiences really help customers feel that the value of what they purchase is not simply a car that gets them from A to B, but something that really makes their lives more entertaining, more comfortable, and more fun.
Describe Unity in three words. I think that's hard, because I think Unity is so many things, but I would say one, fun, because it really is a true delight working with Unity and seeing the outcomes. Two, I would say pipelines, because at the end of the day, you're only as good as your pipelines and how you can bring an idea to life. And third, of course, is scale. The ability for us to tap a whole world's community of Unity developers, assets that are made, understanding and learning from the industry has really helped us.
This transcript has been lightly editorialized for readability.
