After weeks of horrific images and alarming news reports out of Gaza and Israel, it was inevitable that some of the trauma would spill onto social media — and platforms have shown little sense of knowing how to deal with it. But we’re only now getting a glimpse of just how widespread the chaos has been, and how much work platform companies still have to do in digging out of it.
In a recent talk with the Atlantic Council, a member of Meta’s independent Oversight Board gave new details about the weeks immediately following the October 7 attacks. As part of its arrangement with Meta, the Board receives appeals from the company or its users when a moderation decision is believed to be false, or simply raises issues the company can’t resolve. But in the three weeks after the attacks, the board received 20 times more appeals than usual, according to board member Julie Owono.
“The majority of those appeals concern Meta’s Dangerous Organizations policy, but we’ve also seen significant reporting around violent incitement as well as the hate speech policy,” Owono said, adding that the content included “on-the-ground footage, news reports, edited educational videos, [and] blanket statements of support for Israel or Palestine.”
There are plenty of users ready to challenge decisions and escalate through all available channels
This lines up with what users have noticed anecdotally: both a string of decisions cutting against Palestinian interests, and an overall chaos that makes it difficult to tell what the platform policy actually means. News videos have been abruptly hidden by platform algorithms. Prominent accounts have been suddenly locked down, ostensibly for security reasons. Most recently, Meta approved an ad calling for the death of a Palestinian activist — ultimately taking it down when it was reported by a nonprofit group. Images from Gaza are often grisly and upsetting, showing the inherent violence of the conflict itself. Hamas is on Facebook’s official list of dangerous organizations (although notably, it gets the designation from the United States’ FTO list rather than the broader UN consolidated sanctions list), so even basic descriptions of the conflict are subject to takedown on the basis of naming the group. It’s a situation that’s ripe for ambiguous moderation calls, and with tempers high on both sides, there are plenty of users ready to challenge decisions and escalate through all available channels.
Despite great progress in moderation policy over the past 10 years, the crisis is catching the industry largely unprepared. Most trust-and-safety policy work is about establishing specific cases where a particular action is warranted. In 2013, platforms were still mostly caught off guard by nonconsensual pornography; now, every major platform has a formal workflow for reporting that content. “I’m in this photo and I don’t like it” is well-known enough to be a meme. Similar workflows have been developed for inauthentic accounts and manipulated media. The maddening challenge of moderation is always to draw clear lines between what’s allowed and what isn’t — but in these specific cases, we’ve been able to draw those lines.
The conflict in Gaza sits awkwardly between these categories. There are lots of false claims, but it’s difficult to assess when they qualify as misinformation. There are lots of alarming statements — but often, designating them as “hate speech” only muddies the waters further. There are ideas about how to moderate during war, but what’s happening in Gaza is not a conflict between two armies but between a national military, a terrorist group, and a civilian population, with fraught racial and religious divisions between all three. For corporate leaders, the United States’ close ties to Israel make the ongoing war impossible to brush aside. There are simply no rules for how platforms can chart a course through something like this, and little of the prior trust and safety work applies.
Cataclysmic moments have reshaped moderation norms before. In 2010, WikiLeaks challenged how online outlets could oppose U.S. interests, and how far infrastructure companies could go to stop them. In 2016, the scandal around Cambridge Analytica forced Facebook and other platforms to take more responsibility over their own users. As the current tensions reach a breaking point, we may be headed for another such turning point soon.
