Executive Summary
The online games industry continues to fall short in protecting players from hate and extremist content in games. This report focuses on hateful usernames, which should be the easiest content for companies to moderate. Our findings demonstrate that many game companies are failing even the most basic moderation efforts by allowing usernames such as “HeilHitler.”
Following an inquiry from a group of US Congress Members about the proliferation of hate and extremism in online games, all the companies studied in this report provided responses about their efforts to moderate hateful content in their games. Despite game companies having policies prohibiting hate, researchers at ADL Center for Technology and Society were easily able to find usernames in five categories of hate across five popular online multiplayer games
Games function as entertainment and sources of community for millions of people. If the industry continues to deprioritize content moderation, it will send a clear message to users, especially marginalized groups, that games are not safe, welcoming spaces for all.
Key Takeaways
- Despite game companies having policies prohibiting hate, researchers at ADL Center for Technology and Society were easily able to find usernames in five categories of hate (antisemitism, misogyny, racism, anti-LGBTQ+ and ableism) across five popular online multiplayer games (League of Legends, PUBG, Fortnite, Overwatch 2, and Call of Duty).
- Our researchers also found many examples of obvious white supremacist terms in usernames across the online multiplayer games that were investigated. This underscores the fact that none of the game companies investigated as part of this project have policies prohibiting extremist ideologies.
- Usernames are a basic part of any online video game experience. When a player signs up to play an online multiplayer game, they usually have to enter an email address, create a password, and then create a username.
- Usernames are also one of the easiest pieces of content for game companies to moderate, as usernames are evaluated before they are created, are persistent in a game space and are tied to an individual user.
- Additionally, usernames are one of the few elements of online multiplayer games that external researchers can investigate through third-party tools and testing the registration of different terms. This highlights the need for game companies to share data with researchers to allow external evaluations of their stated efforts.
- Our results demonstrate that the game industry is not invested enough in the simplest solutions to address hate and extremism in online games.
Out of the five games examined, Overwatch 2 generated the fewest results when searching for offensive usernames. These results could indicate an effective policy against offensive usernames that could be adapted to other games.
Introduction
Since 2019, ADL Center on Technology and Society (CTS) has conducted an annual survey of hate, harassment, and extremism in online multiplayer games.
Our 2022 survey found that 86% of adults and 66% of teens experience hate and harassment in online games. This has been trending upward every year for the last four years. Additionally, 20% of adults have been exposed to white supremacist ideologies in online games, a deeply concerning spike from 8% in 2021.
In March of this year, CTS published a report analyzing the policies of 12 games, as well as trust and safety priorities of the industry, and found the companies were still falling short. As a result, CTS will be doing deeper dives into how hate manifests as part of different in-game elements of online games. For the first investigation, this report focuses on hateful usernames, which should be the easiest content for companies to moderate.
Usernames are a basic part of any online experience. On social media, users must choose a username for their accounts that displays whenever they post or reply to anyone. This name becomes an identifier unique to that person and represents them.
Online video games also have usernames to identify each unique player. When a player signs up to play an online multiplayer game such as Fortnite, they usually have to enter an email address, create a password, and then create a username. For the most part, players are free to create any username they can imagine unless that exact username is already taken.
These names are then displayed alongside the player in-game chats and usually above their player avatar in the game’s world.
However, offensive usernames are a tool some use to create a hateful and hostile gaming environment. Such names include insults, ethnic slurs, swear words, and other offensive terms.
Academic research on Reddit and the online multiplayer game League of Legends found that offensive usernames correlated with antisocial behavior toward other users and players.
Filtering offensive terms from usernames is one of the simplest steps game companies can take to encourage safer online communities. These systems are entirely text-based and simpler to moderate than voice or in-game text chats.
Many game companies have claimed they are addressing the issue of hateful usernames in their online games. The game companies studied as part of this report responded in detail to a December 2022 letter from Representative Lori Trahan of Massachusetts’ 3rd Congressional District and a coalition of lawmakers that asked game publishers what measures they take to protect players from hate and extremism in online games.
Riot Games stated that they use “filtering technology and name checking system [to] prevent extremists from communicating and using screen names linked to extremist ideology.” Innersloth LLC randomizes usernames for players under the age of thirteen.
Other game companies that mentioned using filtering technology in their games (especially for in-game chat) were Microsoft Gaming, Square Enix, Take-Two Interactive Software, Ubisoft, Valve, and Sony Interactive Entertainment.
CTS’ investigation found that while game companies have filtered out some of the most obvious offensive terms and slurs, they have many holes in their filtering systems. CTS found that in addition to more obvious terms, newer terms and more abstract code words referring to white supremacist ideology are also not being caught by username filtering.
This report is based on CTS’s examination of registered usernames in five popular games. CTS also tested the username registration systems for two of these popular games.
Findings
Tracking Offensive Usernames
The following examples of offensive usernames all have played matches in their respective games, which means that their usernames were seen by other players.
PUBG: Battlegrounds explicitly outlines a policy against usernames in its Rules of Conduct. They claim to forbid usernames that are sexually explicit or obscene, insulting towards different identity groups, or against the social norms of a player’s region. They also state:
In addition, inappropriate names whose spelling has been partially altered or mixed with other letters to render them difficult to discern at first glance will be changed to an arbitrary and different name without prior warning or subjected to penalties.
Despite these policies, CTS was able to find offensive usernames.
Other games also had offensive usernames attacking women, the LGBTQ+ community, and people with disabilities.
For every game except Blizzard Entertainment’s Overwatch 2, CTS found at least one instance of a username with an offensive term in it from the six categories of its 30 sample terms.
Testing Offensive Usernames
Here, CTS tested registering offensive usernames in two games.
Screenshot of offensive username that CTS was able to register using Epic Games’ system. “Day of Rope” is a white supremacist concept.
Call of Duty: Warzone
Call of Duty: Warzone’s registration page allowed 52% of the names we tested. For those filtered out, they explicitly stated that the proposed usernames were inappropriate.
Example of a more recent misogynistic term used by the incel community that is not being blocked by the same registration system.
Fortnite
Fortnite registration, on the other hand, was less clear about whether a term was disallowed or not. Only a handful of offensive terms were explicitly described as “disallowed.” Other offensive terms were classified as “already in use.” This could mean that the username was previously registered and banned, the username was changed, or it is still in use. Without more information from the game company, we cannot clarify any of these potential scenarios except that the username was allowed to be created. Of the usernames we tested, 82% were either allowed to be registered or had been previously registered.
Conclusion
Out of the five games examined, Blizzard Entertainment’s Overwatch 2 by far had the fewest offensive usernames according to the Tracker Network. CTS searched the same terms and found far fewer results from Overwatch 2 users.
This may be due to the fact that of the games we investigated, Blizzard Entertainment’s Overwatch 2 was the newest, having only been released in 2022. It could also be in part because Blizzard Entertainment has invested in online safety for players through initiatives such as its Defense Matrix. In the absence of more information, our results seem to indicate that they may be prioritizing player safety.
For the other four games, the results of this investigation demonstrate they are not doing nearly enough to address hate and hateful ideologies. Preventing offensive terms in usernames requires more than simply keeping a list of offensive terms. Game companies should engage with experts who are aware of the contexts of these terms. Many newer misogynistic terms, for example, originated in extremist incel communities. Given how hateful terms have dynamic meanings that often change depending on contexts and over time, game companies will need to keep track of how these terms are being used by extremists and other hateful actors.
Ultimately, with such dismal results in what should be the easiest content to moderate, this report raises questions about game companies’ efforts to address more ephemeral forms of content such as voice chat, text chat and other user-generated content. If game companies are not investing enough to make the simplest aspects of their games safe and inclusive, what does that mean for their efforts to address hate in more complicated elements of online games?
Methodology
In the absence of more detailed internal information from game companies, CTS used third-party tools and game registration systems to gather data about how these companies filter offensive usernames. In the future, increased transparency in the industry could help these audits and drive improvement of safety practices for the game industry.
Offensive Terms
Using the ADL Center on Extremism’s Glossary of Extremism, the Center for Technology & Society (CTS) compiled a list of over 150 offensive terms. These terms were then sorted into five categories based on identity and affiliation with hateful ideologies: antisemitism, misogyny, racism, anti-LGBTQ+, ableism, and white supremacist terms.
For testing already existing usernames, CTS prioritized 2-4 terms from each category based on their likeliness to be used colloquially. Since the White Supremacist category is the largest and contains complicated, coded terms, its sample contained eight terms. This led to a total of around 30 terms sorted into each category. Any username containing the term was counted as being offensive.
To test registering usernames, CTS used a different sample because it was testing the terms by themselves (as opposed to existing usernames). This sample used the same categories with the following number of terms tested for each:
Antisemitism – 6
Misogyny – 3
Racism – 3
Anti-LGBTQ+1 – 10
Ableism – 2
White Supremacist – 10
The following table shows the categories CTS and created and their definitions.
Games
CTS used the Tracker Network2 to search usernames across five popular games in the Tracker Network’s top 25 games that were also listed among the top eSports games of 2022 by Statista. These five games were also chosen because the Tracker Network had more information for them compared to other games.
League of Legends – Riot Games (2009)
League of Legends is a team-based strategy game where two teams of five powerful champions face off to destroy the other’s base. Players choose from over 140 champions to skillfully take down towers and battle to victory.
League of Legends has a Code of Conduct that gives examples of good and bad behavior. They also explain the consequences of offenses.
PUBG: Battlegrounds – Krafton (2017)
This game is a downloadable online-only multiplayer shooter for Windows PCs (and mobile). Up to 100 players compete in a last-man-standing, kill-or-be-killed battle royale. (Source: Common Sense Media)
PUBG has Rules of Conduct that includes a section on offensive usernames.
Fortnite – Epic Games (2017)
137,786,994 Players Tracked
This game pits up to 100 players against each other in solo, duo, or up to four-player squads to see who can survive the longest against each other on an ever-shrinking map. (Source: Common Sense Media)
Fortnite has a Code of Conduct that explicitly prohibits hate speech but does not outline a username policy specifically.
Overwatch 2 – Blizzard Entertainment (2022)
Overwatch 2 is a hero shooter where players are split into two teams and select a “hero” from a roster of 35 characters. Characters are organized into classes with different abilities to support their team. (Source: Common Sense Media)
Blizzard has an In-Game Code of Conduct that specifically mentions usernames as part of its enforcement policy.
Call of Duty: Warzone – Activision (2020)
52,994,614 Players Tracked
This game is a free-to-play and standalone addition to Call of Duty: Modern Warfare (2019). It is a battle royale-style game: the primary mode has up to 150 players dropped into an enormous map where they work in squads to systematically kill all enemies while gradually being pushed into closer quarters via an approaching poison gas cloud. (Source: Common Sense Media)
Call of Duty has a Security and Enforcement policy that forbids offensive behavior. They also updated their reporting system at the end of 2022. This new system will first be implemented into Call of Duty: Modern Warfare II.
CTS also tested registering usernames for Call of Duty: Warzone (Activision) and Fortnite (Epic Games). These games were picked because they had the simplest registration systems that did not require downloading the game to create usernames. Their registration systems also immediately inform the user if the username is taken or inappropriate (filtered out).
Endnotes
- We over-sampled for this category because of the rise of hate crimes against LGBTQ+ individuals.
- The Tracker Network is a website and app that tracks information about players across a wide variety of popular online games. It tracks user profiles, leaderboards showing who has won the most matches, and special events such as tournaments. It is thus a rich source for looking up existing player names.
