What DSA Article 16 enforcement numbers tell us
There is an ongoing debate about the DSA on whether it has led or will lead to greater restriction of content in the EU. Those concerned with holding big tech accountable will hope that the DSA is compelling platforms to take action on illegal content that they have previously ignored. Those concerned with protecting freedom of expression will be on their guard.
Because the DSA does not define illegal content, or oblige platforms to make their Community Guidelines (CGs) more restrictive, we have looked at Article 16 removal volumes as a ratio of EU CG removals as a proxy measure for content that would not otherwise have been moderated had it not been for the DSA. As a reminder, Article 16 of the DSA obliges platforms to set up an illegal content reporting and moderation mechanism.
To illustrate with a simple example. If a platform made 10,000 CG and 50 Article 16 removals in the EU, we would calculate an Article 16 to CG removal ratio of 0.5%. We assume that those 50 pieces of content would not have been removed without the DSA.
We looked at 5 platforms for this article: Facebook, Instagram, Pinterest, X and YouTube. Here is the summary of the main takeaways:
Pinterest had the lowest Article 16 to CG removal ratio of 0.002%
For Instagram and Facebook, the Article 16 to CG removal ratio was well under 1%, between either 0.1 and 0.3% or 0.03% and 0.05% depending on whether copyright & defamation are included in the Article 16 metric
YouTube was somewhat higher, between 1% or 0.01% depending on whether Copyright removals are included
X was an outlier, with the Article 16 to CG removal ratio coming in at 13%
Methodology
Callouts
The reporting period differs across platforms. The relative, not absolute, values are therefore of interest to us.
Meta, YouTube and X include copyright in their Article 16 numbers, whereas Pinterest do not. We prefer the Pinterest approach because IP and defamation reporting mechanisms already existed on these platforms prior to the DSA and therefore are not novel enforcement categories. For the platforms that included these, we made an adjusted set of calculations without the IP & defamation numbers.
This was more complicated again for YouTube, which does not provide a breakdown of their Article 16 enforcement by category of illegality. We therefore applied an extrapolated value on the basis of the category of incoming reports. We calculated that copyright reports accounted for approximately 86% of incoming Article 16 reports. We assumed a 1:1 report to enforcement category corelation and thus calculated the non-copyright enforcement ratio to be 13% of total enforcement. This is therefore an approximate value.
Pinterest on the other hand include copyright in their CG enforcement figures. We have adjusted this accordingly by removing it in their CG enforcement figures.
For EU CG enforcement across all platforms, we have limited our scope to removal actions at content level, thus excluding content demotions and account suspensions.
Pinterest include a breakdown of proactive and reactive enforcement for EU CG removals. For Meta, YouTube & X, it appears to include only proactive enforcement. We have adjusted this accordingly.
Note that proactive enforcement often accounts for over 95% (sometimes 99%) of total enforcement. Whether reactive enforcement in included or not should therefore have a negligible effect on the datasets.
Meta calculations
Time period: 1 October 2023 to 31 March 2024
For Article 16 removals, we used Table 15.1.b.(1) - Number of notices submitted in accordance with Article 16 DSA, by type of alleged illegal content and actions taken for Facebook.
For total CG removals in the EU, we used the values in Table 15.1.c.(1) - Number of organic content removal measures in the European Union for Facebook.
The same applies to the Instagram report mutatis mutandis.
Pinterest calculations
Time period: September 25, 2023 to December 31, 2023
For Article 16 reports, we used the Report Outcomes table for European Union Illegal Content Reporting Form Reports.
For total CG removals, we used the total number of deactivate actions indicated for EU Member State users' Pins.
X Calculations
Time period: 21 October, 2023 to 31 March, 2024
For Article 16 removals, we used the total of the two tables in the REPORTS RESOLVED BY ACTIONS TAKEN ON ILLEGAL CONTENT section:
Art. 15.1.b & c: Automated Content Deletion Actions Taken on Reported Illegal Content - 21/10/23 to 31/3/24
Art. 15.1.b: Manual Content Deletion Actions Taken on Reported Illegal Content - 21/10/23 to 31/3/24
For CG removals, we used the table Art. 15.1.c: TIUC Terms of Service and Rules Content & Profile Removal Actions - 21/10/23 to 31/3/24.
YouTube calculations
Time period: 11 September 2023 to 29 February 2024
For Article 16 removals, we used Table 2.3.1: Number of actions taken in response to Article 16 notices, by service and basis of the action 1. The YouTube enforcement figures are hugely influenced by the weight of Copyright enforcement, which we have provided adjusted values for.
For CG removals, we used Table 3.1.1.h: Own initiative actions taken on YouTube, by type of illegal content or violation of terms and conditions
Data
| Platform | EU CG removals | Article 16 removals | Article 16 removals (Adjusted for IP removals) | Art 16 to CG removal ratio | Art 16 to CG removal ratio (Adjusted for IP removals) |
|---|---|---|---|---|---|
| 14,786,798 | 338 | 338* | 0.002% | 0.002%* | |
| 69,802,131 | 122,351 | 32,434 | 0.175% | 0.046% | |
| 37,039,411 | 98,844 | 11,375 | 0.267% | 0.031% | |
| YouTube | 38,945,694 | 362,988 | 48,277 | 0.932% | 0.124% |
| X | 483,591 | 63,174 | 43,930 | 13.064% | 9.084% |
* The Pinterest numbers for Article 16 removals did not include IP or Defamation violations from the outset. No downward adjustment was therefore necessary for further comparison.
Conclusions
These numbers tell a different story for each platform.
For Pinterest, they suggest that the gap between their Community Guidelines and EU law is minimal and that their CG enforcement is already extremely effective at tackling all kinds of illegal content.
For Meta, it’s a similar case, albeit there is a greater gap than for Pinterest, it is somewhat smaller than for YouTube.
X is the outlier here with the Article 16 to CG removal ratio coming in at 13%. This is most likely due to the more libertarian and permissive policies adopted by the platform since its takeover by Elon Musk.
There are other variables at play here too, including ease of access of the Article 16 reporting mechanism or the internal operational structure, which can qualitatively influence action these ratios one way or the other. We may look at this in more detail another day. There are many other VLOPs and it was beyond our capacity to repeat this study across all platforms.
If you found this post insightful, please share. If your team is struggling with setting up illegal content reporting mechanisms, don’t hesitate to get in touch.
Liam Melia is the Managing Director of Pelidum Trust & Safety
© Pelidum Trust & Safety 2024