How Hi-Rez tackles toxic behavior at massive scale

Building long-lasting, inclusive communities is at the heart of Hi-Rez’s mission. “Our biggest priority right now is understanding the needs from our players in terms of trust and safety,” says Jones. But toxic behavior and even harassment are pervasive in multiplayer games, and detecting problems in a community of millions would be challenging for a team of any size.

Hi-Rez uses Safe Voice’s predictive analytics to perform both acoustic (tone of voice) and semantic (meaning of words) analysis on in-game voice communications. The system adds AI-generated labels that make it clear why a session is being reported, speeding up moderators’ workflows.

Jones has 12 years of experience managing game communities, and he calls voice chat the “wild West” in the trust and safety space. Few tools have the ability to parse and visualize in-game voice data, and Jones describes other solutions as “egregiously expensive and not realistic” for small and mid-size studios.

With Safe Voice, reported instances are classified using metrics like toxicity score, disruption score, overall risk, and whether at-risk players were impacted in a given incident. This helps Jones’s team to “identify disruptive behavior and proactively address minor toxicity so we can rehabilitate players instead of cycling them out.”

Ensuring an optimal threshold for identifying toxic or disruptive behavior is important for Hi-Rez. To strike the right balance between fun and safety, Hi-Rez fine-tuned Safe Voice using its toxicity and disruption filters.

“False positives result in player appeals, which create more work for us. Initially, we had profanity contribute to toxicity scores, but most of the time we found that the incidents weren’t actionable,” explains Jones. By configuring Safe Voice to match the culture of their game, Hi-Rez reduced false positives by 90%, which improved incident prioritization and response times.

Not all disruptive behavior is verbal. Loud noises, music, and even pet sounds can take away from the fun of your game. When Hi-Rez first started using Safe Voice, they realized that “a lot of background noise was coming through people’s microphones.” During the 45-day monitoring period, the team identified nearly 3,000 sessions that were negatively impacted by background noise.

Armed with this insight, Jones and his team started thinking about solutions. “It made us consider implementing noise reduction in the game,” he says. “We wouldn’t have known that this was an issue without Safe Voice.”

Just a few weeks of using Safe Voice shed new light on issues impacting the Rogue Company player community. As Jones puts it, the tool “illuminates what was previously this dark space. We can now see what’s there and take action to improve.”