jugendschutz.net Annual Report 2024Sexualized violence and hate content: Dangers for children and young people online are increasing

According to the annual report by jugendschutz.net, the successful elimination of youth protection violations is hampered by structural deficiencies in the precautionary measures of many digital services. The central problem remains the lack of age verification, without which children and young people are hardly protected on popular platforms - for example from extremist influence or sexual boundary violations.

Added to this is the growing influence of generative artificial intelligence, which is also fueling the spread of problematic content. The focus of providers here is on innovation, while safety for young users is not a priority.

Although 99% of the content reported by jugendschutz.net was deleted, it is clear that simple user reports are not treated with the seriousness they deserve by the platforms. To check this, jugendschutz.net initially reported violations in the area of extremism via the normal reporting channels open to all users. This made it impossible for the platforms to recognize that jugendschutz.net was actually behind the report. After seven days, the content was reported again if it had not already been removed after the first report. This time, however, it was an official report from jugendschutz.net. The results of the test on YouTube were particularly striking: Only 6% of user reports resulted in deletion. After the official report by jugendschutz.net, the deletion rate jumped to 96%. Other platforms such as Instagram (17% / 89%) and TikTok (27% / 94%) also deleted significantly less content when the reports supposedly came from regular users.

"Social media continues to be a gateway for sexual harassment, violence and extremism", says Stefan Glaser, Head of jugendschutz.net. The increased targeting of young people by extremist groups, including in the gaming sector, is particularly alarming: "Right-wing extremists and Islamists use TikTok, Instagram and the like to lure people in with hip appearances and topics that are close to the lives of young people. They use Discord to reach young people with an affinity for gaming and introduce them to extremist ideas." At the same time, technical innovations in the field of AI are exacerbating existing risks. "Unfortunately, it's now child's play to turn everyday photos into nude images. Deepnudes are then used to bully or blackmail - a perfidious dimension of digital violence," warns Glaser. jugendschutz.net's research has shown that operators are improving their protection concepts in some cases. "Ultimately, however, age-differentiated measures can only be effective if the age of the user is also checked," says Glaser.