Real411 develops new tool to track incitement on social media

The Real411 teams have been kept busy with all manner of content, from claims about the elections being rigged to claims that people need to take their own black ink pens to vote, as the IEC will be using “evaporating ink”, suggesting that after marking the ballot, voters’ Xs will disappear, leaving a blank ballot for bad actors to then mark as they choose. It’s just wrong.

One of the key lessons learned from the 2019 national elections is that online harms often reinforce each other. Women journalists are frequently attacked online — harassed and threatened while disinformation about them is peddled. Less focus has been given to the issue of incitement on social media.

Incitement is a criminal offence in South Africa, legislated against in a few Acts including the Cybercrimes, and Riotous Assemblies Acts. We have made use of these in forming the criteria for assessment on Real411, where for something to be incitement it had to meet all the criteria.

In February, the South African Human Rights Commission released its report on the inquiry into the July 2021 unrest. A key finding was that, “through the creation and dissemination of inflammatory content, social media amplified grievances, stoked fear and anger, and mobilised individuals towards disruptive actions”.

At the same time, we had been observing trends in how bad actors on social media would know where the lines were on various platforms to not cross them, which would ensure their content remained online and would not be removed or deprioritised.

Given the risk that incitement online posed, we wanted to get a sense of whether we could spot it online using AI, and then see if any trends emerged.

While some of the platforms, including Google, Meta and TikTok, act against clear instances of calls to violence, we wanted to get a sense of two additional elements. The first was to see how much content was inciting violence, and the second was to look at those instances where the content had elements of incitement but did not meet all the criteria — it stopped just short of calling for violence that would cause public harm.

It became clear that there are varying definitions of “incitement”.  We found that incitement tended to be spoken of not in its own category but most commonly in relation to hate speech, of which it forms a component. We worked with a diverse group of experts, with legal, media, technology and linguistic expertise. The tool we developed is now live in beta phase and can be seen here.

We are still training the model. In this first iteration, we hope to highlight and raise awareness of online incitement to act as an early warning system and to help identify terms and phrases and possible networks.

This project forms part of MMA’s broader work around combating online harms. Although Real411 has been the official platform to report election-related disinformation during the election period, the platform itself operates year-round, dealing with complaints that fall into four categories: disinformation, harassment, hate speech and incitement to violence. If you come across content on any digital platform that you suspect falls into one of these categories, please report it to Real411.  DM



Download the Real411 App on Google Play Store or Apple App Store.

William Bird is the director of Media Monitoring Africa (MMA). Thandi Smith heads the policy and quality programme at MMA, a partner in the 411 platform to counter disinformation.