Roblox makes three of its AI safety tools open source
Roblox is open-sourcing three of its AI safety models via the Robust Open Online Safety Tools Model Community (ROOST). Read more
Roblox has released three of its AI safety models as open-source tools through the Robust Open Online Safety Tools Model Community (ROOST). ROOST is a nonprofit organization established in 2025, co-founded by Roblox, Google, OpenAI, and Discord. The company contributed its PII Classifier, Roblox Sentinel, and the latest voice safety classifier to ROOST.
The PII Classifier detects attempts to share or request personally identifiable information, even if it uses misspellings or coded language. Roblox Sentinel identifies early signs of possible child endangerment, while the voice safety classifier moderates real-time voice chat and incorporates feedback from ROOST's community. Additionally, Roblox is releasing a new evaluation dataset to assist other companies in assessing the effectiveness of their safety tools, which now supports 30 languages and eight violation categories.
Roblox's vice president of trust and safety engineering, Naren Koneru, explained that sharing these models will provide other platforms with a strong foundation for training and refining their moderation tools. The company hopes to benefit from the learnings and feedback of other organizations as they share their progress in automated detection.
Recently, the US Senate Judiciary Subcommittee on Crime and Counterterrorism began an investigation into Roblox, accusing the platform of prioritizing revenue and engagement over child safety.
Written by urgent.news from GamesIndustry's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
