Developing Grounded, Human Rights-Centered Responses to Deepfakes, Synthetic Media, and Audiovisual Generative AI

Significant lessons can be learned from centering civilian and citizen concerns—including those of human rights defenders and journalists, globally—in the prioritization of risks, threats, and potential solutions in the areas of deepfakes, synthetic media, and audiovisual generative artificial intelligence (AI). Worldwide, these individuals and communities already confront harms similar to those emerging in the 'age of synthetic media'. However, despite being most at risk, these individuals and communities are marginalized from decisions on these new technologies.

Launched in 2018, WITNESS's 'Prepare, Don't Panic: Deepfakes and Synthetic Media' initiative has aimed to intervene early in the synthetic media ecosystem, focusing on technical infrastructure, emergent tools, digital literacy efforts, and policy and legislative aspects. The work is based on extensive research, industry consultation (including workshops in Europe, South Africa^{Footnote 146}, Brazil^{Footnote 147}, Southeast Asia^{Footnote 148}, the United States^{Footnote 149}), and numerous online workshops and consultations across global geographies^{Footnote 150}.

Threats and Risks from a Civil Society Perspective

Civil society actors, through the past five years of WITNESS consultations, consistently identify a set of existing harms and potential risks from synthetic media. Women are consistently identified as being particularly vulnerable to threats from synthetic media because of how the technology has enabled new forms of gender-based violence.

Synthetic media, or the existence of it, is used to exercise plausible deniability and dismiss demonstrably true evidence by claiming it is false (also known as the ‘Liar’s Dividend’), or to claim that all content cannot be trusted—often to discredit journalists, activists, and civil society organizations more broadly, along with the trustworthy content they put out to the world. Research participants expressed concern on the use of such claims (or fears) to justify the enactment of laws that limit speech more broadly.

The threat of synthetic media to spur misinformation and incite violence is consistently noted —particularly within existing vectors of rapid spread such as messaging apps—as well as how this can be used by foreign and domestic actors to target groups and communities who are already vulnerable based on ethnicity, religion, political identity, professional role(s) and/or other characteristics.

All of these threats are consistently connected to the existing challenges of media literacy, under-resourced journalistic capacity, and limited access to detection and authenticity tools for both critical civil society actors and individual citizens.

Workshop participants identified that these threat vectors combine with existing threat patterns directed towards civil society and citizens by their own governments in the context of closing civil society space. For example, spreading disinformation targeting civil society, surveilling and harassing of journalists and human rights defenders, and attempting the criminalization of their activities.

In the past year, as generative-AI based synthetic media tools have become more accessible, easier to use and more personalizable, more people have had the ability to engage with them. They have been able to imagine—or experience—how the tools could impact their lives. This shift has resulted in an emerging reevaluation of the risks and potential harms.

As people experimented with synthetic media tools and realized how easy they were to use to create individual content items, produce variants on content, and produce images of real-life events (with limited input data), the challenge of an information ecosystem flooded with synthetic content came up more regularly linked to the inadequacy of the volume of, for example, fact-checking responses. Participants placed these in critical contexts such as elections and public health crises^{Footnote 151}.

Principles for Building Better Civilian Resiliency

In terms of building civilian resiliency to AI-based manipulation and synthesis, a number of core principles emerged through this research.

Prioritize: 1) people globally facing similar harms; and 2) journalists and civil society who support a reliable, trustworthy information ecosystem

A response to deepfakes and synthetic media, as well as the expansion in availability and ease of creation of audiovisual generative AI and deepfake technology requires attention to who is most at risk from both targeted attacks as well as a broader undermining of trust in critical content. In many cases, these same communities and stakeholder groups have already experienced related harms from previous technologies. For example, globally women journalists and public figures are targeted with non-consensual simulated sexual images, while human rights defenders and investigative journalists consistently face claims that their documentation and investigations have been falsified.

Similarly, these same constituencies already face real-world constraints on their capacity to respond. For instance, local journalists, local elections officials, and female and LGBTQI-identified community-level political figures often find themselves targeted, under-resourced, and over-burdened.

Avoid de-historicizing or decontextualizing synthetic media

Although synthetic media is an emerging technology, the threats it poses are not new. As noted above, existing experiences, particularly of vulnerable populations, critical civil society, and media intermediaries, should inform responses to the threats and opportunities of synthetic media. Marginalized populations know the ways in which they are targeted with disinformation, and community-based response strategies and fact-checkers have experience addressing more traditional methods of video and audio manipulation, or ‘shallowfakes’; while social media platforms are already grappling with how to handle satire (a common deepfake usage) on a global level.

Place firm responsibility across the pipeline of foundational model and tech builders, tech deployers, content creators and content distributors (media and social media)

Any solutions require careful attention across the pipeline of how synthetic media is made, from foundational model and tech builders, to the deployers of technology and content distributors. Responses should not place the burden or pressure on end-users to identify synthetic media or disclose their personal usage, in the absence of broader responsibility throughout the pipeline of how synthetic media is created.

For instance, it is not viable to double-down on a media literacy strategy focused on civilian forensic analysis of video. An example of this is the reliance on or promotion of tips that encourage someone encountering an image in their timeline to attempt to spot a potential visual glitch such as a distorted hand, created through the generative process. These tips rely on the current algorithmic ‘Achilles Heel’ and are often quickly remedied by technical progress.

Governments, social media platforms, technology companies, and news organizations all have a role in the development of mechanisms (such as regulation, policies, functionalities, processes, etc.) that proactively tackle threats without placing the responsibility on content creators or consumers alone, and locating responsibility (where relevant) with upstream stakeholders.

Developing a human rights-based technical infrastructure, norms, consistent global platform policies, and laws and regulations is key.

Goverments and regulators can support a range of options at the technical and policy levels that help to establish clear human rights-based guardrails, mandate rights protections, and also pay close attention to critical rights issues around privacy and freedom of expression.

Actions to Support an Informed Digital Citizenry

These prioritized solutions reflect outcomes from WITNESS’s research and require implementation that takes into account the factors identified above.

Media and digital literacy will not be enough, but are still as necessary as ever

Supporting media literacy more broadly is a critical component of societal and governmental responses. This is especially true considering that society is in the early stages of the use of synthetic media and deepfakes, and that so-called “shallowfakes”—where media is decontextualized, lightly edited or miscaptioned—are currently far more prevalent than synthetic content.

Techniques and approaches to synthetic media should not be developed in isolation from broader media literacy approaches or from approaches relevant to shallowfakes. For example, the SIFT approach^{Footnote 152} that focuses on the core principles of Stop, Investigate the source, Find alternative coverage, and Trace the original context, is an applicable methodology across broader media literacy, shallowfakes and emergent deepfakes. Media literacy campaigns need to be framed within the broader context of mis- and disinformation in an effort to promote critical consumption of content online. Media literacy campaigns should not focus on current technical flaws in particular generative techniques—or example, the well-known and now discredited tip that face-swap deepfakes do not blink—but on broader principles, and on the use of contextually appropriate detection and authenticity tools as they become available.

One key approach to consider in media literacy campaigns is to avoid adding to the existing hype around generative AI and synthetic media, and mitigating their impact in reducing the ability to trust content. Particularly for vulnerable communities and critical civil society voices, claims of ‘it’s a deepfake’ and the broader ‘nothing can be trusted because anything can be falsified’ are already growing in prominence, and media literacy campaigns should be calibrated to the scale of the synthetic media problem and avoid alarmist rhetoric that compounds this issue.

In tandem with media literacy approaches aimed at the general population it is important to focus on ‘the media’s’ literacy, and journalism’s contribution to neither providing a simplistic educative response focused on visual tips, nor contributing to the panic cycle that is weaponized in some contexts against critical societal voices.

Detection tools should be accessible to those that need it most

Detection tools are part of a solution. In general, current detection tools are not reliable at scale, and require expert input to assess their results. In a number of global cases, the use by the general public of detection tools available online has contributed to confusion and increased doubt around real footage rather than contributing to clarity^{Footnote 153}. In the short-term, however, critical gaps exist in supporting better capacity and access to tools for journalists and fact-checkers, as they look to debunk realistic forgeries or dismiss claims that genuine journalistic multimedia content is fake.

Accessibility in these conditions extends beyond technical access to support in how to effectively use the detection tool(s), and to effectively communicate results to stakeholders.

While platforms can play a role in moderating synthetic media content, this should not be an automated process of unnuanced synthesis detection given the unreliability of existing detection models. The reality must be acknowledged that not only is most synthetic content personal communication or non-malicious/harmful, but that content will increasingly be a complex mix of synthetic and non-synthetic, and that detection efforts will have to accommodate for this. However, there is the potential for platforms to provide signals that could support both civil society and media entities doing analysis, as well as the broader media literacy noted above.

Verifiable provenance and watermarks can provide signals to support informed digital participation, but human rights and accessibility concerns should shape the infrastructure and tools

A range of initiatives are exploring how to provide authenticity and provenance signals to consumers of media (for example, initiatives such as the Coalition for Content Provenance and Authenticity, C2PA.org and the Content Authenticity Initiative, contentauthenticity.org). While there are a number of proposals for incorporating watermarking technologies into AI-generated technologies, all of these approaches rely on participation and integration across the pipeline of responsibility (outlined above) from foundational model developers, tool developers, social media platforms, and major news media outlets. They all hold a key responsibility to enable disclosure of how media encountered by citizens is created. Buy-in from social media platforms, major news media outlets and AI model and tool developers is essential as they play a key role in guaranteeing that the verifiable provenance or watermarks are part of, or attached to, the audiovisual content from early on in its lifecycle, but also in making sure that their users and viewers are effectively informed about the nature of the content they are consuming.

Within this context, a core responsibility for democratic government is to ensure that these technologies are deployed with privacy and access centered, and where necessary legislated. Work by WITNESS has identified a range of human rights concerns that are related to authenticity, provenance, watermarking and disclosure approaches—among the most prominent are ensuring that media provenance is not intrinsically or de facto connected to personal identity, and ensuring global stakeholder input^{Footnote 154}. As with synthetic media developers, the companies, organizations and governments behind initiatives promoting the use of provenance and watermarking technologies have a responsibility to assess these technologies for their potential to cause harm^{Footnote 155}.

Synthetic media content moderation at scale still needs contextualized policies and local expertise to deal with threats from synthetic media and protect speech

Synthetic media will increase the amount of content created and shared online. Automating moderation, including by leveraging AI (e.g., detection tools, or provenance and watermarks), will be a facet of responses, however, these policies should be designed alongside impacted groups and based on human rights principles. In addition, there should be clear processes for local experts and experience to be involved in the moderation loop.

One key area of concern is to preserve options for the frequent satirical and parodic uses of synthetic media^{Footnote 156}, while recognizing that this is a contested grey area and one where ‘gas lighting’ claims of humour are made around actual harmful or malicious content.

Footnote 146

What We Learned from the Pretoria Deepfakes Workshop (Full Report) - WITNESS Blog. Available: https://blog.witness.org/2020/02/report-pretoria-deepfakes-workshop/.

Return to footnote146 referrer

Footnote 147

WITNESS Media Lab | Deepfakes: Prepare Now (Perspectives from Brazil) - WITNESS Media Lab. Available: https://lab.witness.org/brazil-deepfakes-prepare-now/.

Return to footnote147 referrer

Footnote 148

WITNESS Media Lab | Deepfakes: Prepare Now (Perspectives from South and Southeast Asia) - WITNESS Media Lab. Available: https://lab.witness.org/asia-deepfakes-prepare-now/.

Return to footnote148 referrer

Footnote 149

Deepfakes and Disinformation – MediaJustice. Available: https://mediajustice.org/news/deepfakes-and-disinformation/.

Return to footnote149 referrer

Footnote 150

For reports wit.to/Synthetic-Media-Deepfakes.

Return to footnote150 referrer

Footnote 151

WITNESS, Fortifying the Truth in the Age of Synthetic Media and Generative AI, May 2023, https://blog.witness.org/2023/05/generative-ai-africa/.

Return to footnote151 referrer

Footnote 152

See https://hapgood.us/2019/06/19/sift-the-four-moves/ and https://ctrl-f.ca/.

Return to footnote152 referrer

Footnote 153

https://blog.witness.org/2021/07/deepfake-detection-skills-tools-access/.

Return to footnote153 referrer

Footnote 154

Ticks or It Didn’t Happen: Confronting Key Dilemmas in Authenticity Infrastructure for Multimedia https://lab.witness.org/ticks-or-it-didnt-happen/ and ‘Deepfakes, misinformation and disinformation and authenticity infrastructure responses: Impacts on frontline witnessing, distant witnessing, and civic journalism’ https://journals.sagepub.com/doi/10.1177/14648849211060644.

Return to footnote154 referrer

Footnote 155

https://blog.witness.org/2021/12/witness-and-the-c2pa-harms-and-misuse-assessment-process/.

Return to footnote155 referrer

Footnote 156

Just Joking: Deepfakes, Satire, and the Politics of Synthetic Media, https://cocreationstudio.mit.edu/just-joking/.

Return to footnote156 referrer

Page details

2026-09-03

Language selection

Search

Developing Grounded, Human Rights-Centered Responses to Deepfakes, Synthetic Media, and Audiovisual Generative AI

Page details