Discord is going to give out warnings instead of permanent bans

[Collection]

Illustration: Alex Castro / The Verge

Discord is overhauling the way it moderates its platform with a new warning system and teen safety assist feature. Both are part of a round of feature additions coming to Discord in the coming months, including an in-app shop for Discord members, a new Midnight dark theme on mobile, and new ways to launch apps.

The new Discord warning system has been totally revamped to be far more transparent, educating Discord users how they’ve broken rules and are restricted from parts of the service rather than permanently banning them. “The new system gives users more room to learn from their mistakes and correct misjudgments,” explains Savannah Badalich, Discord’s senior director of policy, in a briefing with The Verge. “We’re moving away from permanent bans to one-year temporary bans for many violations, except for violations that are extremely harmful.”

Image: Discord

The new Discord warning system.

In the coming weeks, Discord will start to limit features for rule breakers, instead of banning them outright. If a Discord user violates the rules, then they’ll be met with a DM from Discord letting them know about the warning or violation and what action Discord is taking. So, if a Discord user uploads an image that breaks the rules, they might temporarily take away the ability to post images.

“We think we’ve built the most nuanced, comprehensive, and proportionate warning system of any platform,” claims Badalich. There’s a new account standing section in Discord that looks a lot like Microsoft’s Xbox strike system that offers transparency into any violations on an account. Unlike many other platforms, Discord isn’t taking a “three strikes and you’re out” approach here, instead trying to educate users to improve their interactions online. “We believe people, especially teens, have the ability to change when given the opportunity, so we want to offer those opportunities,” says Badalich.

Alongside this new warning system, Discord is also launching a teen safety assist feature. It’s designed to protect teens through proactive filters and alerts that will be enabled by default. In the coming weeks, Discord is rolling out a sensitive content filter that will be enabled by default and blur sexually explicit content in DMs and in servers. It will also be available for adults to opt in to and can be disabled by both in the settings section.

Image: Discord

Discord will blur sexually explicit images by default for teens.

This means that Discord image attachments will be scanned to detect explicit content. This will likely generate privacy concerns, particularly as teens are involved. While Discord is experimenting with encryption for voice and video calls, the service doesn’t currently use end-to-end encryption for messages like Apple, WhatsApp, Signal, and many others do.

Discord argues its scanning isn’t an invasion of privacy because it’s using AI models instead of humans. “It allows us to deploy more technology to identify problematic content, but it’s in a way that doesn’t feel like it’s a violation of privacy,” explains John Redgrave, Discord’s vice president of trust and safety, in an interview with The Verge. “This is our way of saying we’re not going to dramatically invade everyone’s privacy, while also providing tools that enrich people’s experience from a safety perspective.”

Redgrave joined Discord two years ago after it acquired Sentropy, the company he co-founded to work on AI tools that detect harassment and abuse online. Discord is planning to expand these AI models beyond just blurring. “It gives us a mechanism by which we can introduce additional models in the future that can protect against other forms of challenging content,” explains Redgrave, who says Discord is also working on a grooming model to detect sexual exploitation on its service.

Apple was forced to drop some controversial child protection features last year after privacy outcry but eventually rolled out a similar opt-in feature as part of its Family Sharing setup. Apple can scan incoming and outgoing pictures for sexually explicit material to children’s accounts, blurring images if it detects something sexually explicit.

Discord is also rolling out a safety alert to teens, so when they receive a DM for the first time, they may also get a safety alert with links to block the user or safety tips to safeguard themselves.

Image: Discord

Discord’s new store is coming to everyone soon.

Improved warning systems and safety features aren’t the only things coming to Discord in the coming weeks / months. “We’re making the [mobile] dark theme even darker and we’re adding a Midnight theme which is pure black,” explains Peter Sellis, Discord’s SVP of product. The Midnight theme has been available as a hidden AMOLED optimized mode in Discord’s mobile apps for months, and Sellis claims “Midnight actually saves battery life on compatible devices by turning off the all-black pixels.”

The search functionality on Discord mobile is also being improved soon with tappable search filters and an improved notifications tab with an auto-clear feature. Discord mobile is also getting a new Remix feature this week that lets you remix images into memes and share them on Discord.

If you’re interested in avatar decorations and profile effects, Discord’s in-app store is arriving for all users soon. It offers a bunch of decorations for profiles so your avatar can have an animation over it or people can preview your profile and see effects. Nitro members already have access to the Discord Shop and get a discount on avatar decorations and profile effects.

Last but not least, Discord is also rolling out some improvements for apps and developers. The premium app subscriptions that were previously US-only are extending to the UK and Europe this week. Discord is also experimenting with making apps usable in more places throughout its main app and looking at ways to let Discord users use apps in a server without them having to be added by an admin.