Content moderators pay a psychological price to keep social media clean. We should help them

Content moderators are the unsung heroes of the internet. They work in a growing field that supports today’s social media infrastructure. But keeping us safer, by constantly seeing and filtering out the worst content online, has detrimental consequences. Can people really cope with this constant barrage of horror?

One of the main textbooks for psychologists, the DSM-5, includes “indirect exposure to aversive details” in the category of post-traumatic stress disorder. In extreme circumstances, this can lead to what is often referred to as ‘secondary trauma’.

Secondary trauma can occur when people, such as first responders, attend to victims in distress. It can also happen after watching or reading distressing content. Those who work in content moderation for social media companies, medicine or psychology, and social work, among others, can all relate to how pointless gruesome and memorable cases can be.

Personally, I had flashbacks following my work as a criminal psychologist. The most intrusive were after an instance where I had unusual access to video content, including enlarged shots of very moving victim impact statements.

Among the first to study the specific impact of violent videos was a team led by stress and trauma researcher Arija Birze. In 2022, they published the results of two studies in which police officers, lawyers, judges, psychiatrists, clerks and court reporters were interviewed.

In the first study, they found that, in the criminal justice setting, violent videos increase “new emotional closeness.” It’s harder not to be affected when you can see every second of emotion in hyper-realistic detail. Indeed, compared to photos, being able to see dynamic facial expressions leads to greater activation of the parts of the brain responsible for empathy and emotional memory.

They also found what they called “perpetual visibility”. Before cameras and high-resolution images, most intimate details of a crime would have been unknowable, imperceptible, or fleeting. Details like the look on an attacked victim’s face were previously limited to our imagination. But now we sit in the front row, sometimes even closer than the real witnesses.

More like this

The researchers also found that people were often blinded by the violence. It may not be possible to be fully prepared for graphic images, even if you know they are coming. Added to this is an almost total lack of training for many people struggling with these shocking videos.

It makes sense that the psychology of victims and perpetrators should take priority, but the broader psychological fallout of these crimes must also be considered. Especially because criminal justice and content moderation actors often need to watch video evidence over and over and over again.

Unfortunately, in a second study, researchers found that organizations had failed to keep up with the exponential increase in disturbing video content. Criminal justice professionals generally only receive help when they are so affected that they can no longer do their job. Many do not even receive help then.

Because of this, people rely too heavily on co-workers who are struggling themselves, which can create what researchers call intense “traumatic contagion.” This means that some traumatized people amplify the suffering of others.

Specific interventions have also been developed for content moderators, with mixed success. For example, in 2022, a group of researchers added positive content to breaks. Between the distressing posts, they were shown photos of cute baby animals and awesome landscapes. These types of images have been shown to decrease stress in other research settings, but here it had the opposite of the intended effect. This could be due to the “emotional contrast”, the happy images made the negative even worse.

It appears that the negative effects of violent content are cumulative, and compassion fatigue can also develop over time. This is when people have spent so much time helping to protect others that they become exhausted and their emotions go flat.

For professionals who regularly deal with distressing video content, in addition to organizations providing formal support, taking a break from content can help. It is also important to zoom out regularly. People who focus on general aspects of crimes and content, versus specific details, are less likely to have flashbacks.

More importantly, as technologies develop and new ones emerge, research must keep a constant eye on how to keep those who protect us psychologically healthy.

Learn more about crime:

Leave a Reply

Your email address will not be published. Required fields are marked *