Disinformation, Deepfakes, and the Human Response

The realism and vividness of deepfakes makes them unusually effective at depicting alternative people and facts. People share deepfakes not necessarily because they believe them, but because they want to reinforce their identity and social position. Deepfakes rarely change minds; the threat they pose is to radicalize by sowing chaos and confusion. The following reviews the consequences of deepfakes in both the social sphere and people’s private lives and suggests potential interventions to reduce these negative consequences^{Footnote 61}.

Deepfakes are hyper-realistic digital impersonations or falsifications of images, video, and/or audio created through neural networks using machine learning models called generative adversarial networks (GAN). They appear in a wide range of contexts, from arts and entertainment to advertising, and education. The most frequent application of deepfakes is in pornography; as of October 2019, 96 per cent of deepfakes on the internet were pornographic^{Footnote 62}. Nevertheless, it is deepfakes’ actual and potential contribution(s) to the epidemic of fake news that has engaged academia and the media, although they also pose a number of social threats^{Footnote 63}. Images and videos carry meaning by appearing to represent real life directly. Numerous deepfakes have appeared of politicians making claims that contradict their actual position, like the one uploaded to a hacked Ukrainian news website that depicted Volodymyr Zelensky, President of the Ukraine, telling his soldiers to lay down their arms^{Footnote 64} or Barack Obama using profanity towards Donald Trump^{Footnote 65}. If such deceptions go viral, they could have an irreversible effect on world affairs.

Humans process visual data naturally and thus fluently^{Footnote 66}, and people believe what they see^{Footnote 67}. Moreover, the detailed imagery of deepfakes has the potential to prime psychological proximity. Concrete misinformation (including disinformation) primes participants to think of events as being nearer and more probable, increasing their perceived threat^{Footnote 68}, and likelihood of news about them being shared^{Footnote 69}.

How Effective Are Deepfakes?

Studies disagree about whether people are able to discriminate deepfakes from real images. When compared to traditional sources of fake news, such as text or audio, some studies have found deepfakes to be no more credible or effective at implanting false memories than their counterparts^{Footnote 70}^{Footnote 71}. However, while these studies are recent, the technology is changing so quickly that the studies did not deploy the most recent AI-based image-creation technologies available online. Worse, some studies of deepfake credibility use only a single video^{Footnote 72}.

Lago et al. (2022) reports that newer AI-synthesized images are perceived as real^{Footnote 73}. Indeed, synthetic faces generated by the most state-of-the-art GANs were judged as more real than actual real images, pointing to the potential of deepfakes to simulate reality and circumvent the eerie, unsettling feeling that arises when humanoid robots or computer-generated images are too close to the real thing (the “uncanny valley” effect)^{Footnote 74}. Further, Köbis et al. (2021) show that people cannot reliably detect deepfakes, and that neither raising awareness nor introducing financial incentives improves their detection accuracy^{Footnote 75}. An even more recent study found that deepfake videos are both more believable than fabricated images and text and that people are more likely to engage with them^{Footnote 76}. Rapidly evolving GAN technology will soon render deepfakes indistinguishable from genuine content if it has not already done so.

Do People Care About Accuracy?

Information veracity is not a deciding factor when users choose to share content online^{Footnote 77}. Even when users can identify deepfakes as untruthful, they still might share them within their social circle. Indeed, Vosoughi et al. (2018) found that fake news is diffused faster and further online than factual information^{Footnote 78}.

A hint about why people share and view deepfakes can be found where they are most common. The term “deepfake” was coined by a Reddit forum created for sharing pornographic videos of women whose faces were synthetically swapped for those of others, mostly celebrities^{Footnote 79}. Consumers of pornographic deepfakes are unlikely to be fooled by the imagery they are watching as the website or video title is typically marked as “fake”; there is no pretense of truth. Thus, viewers of pornographic deepfakes obtain whatever benefits they get despite or because of their knowledge that what they are watching is fake. This could also be true of many political deepfakes. Other uses of deepfakes similarly do not depend on their veracity, such as their use(s) for artistic and educational purposes.

Most deepfakes on social media will exist for the same reason as fake news, to attract ‘clicks’. To achieve this, fake news exploits two properties that engage people’s attention: novelty and negativity^{Footnote 80}. Sharing novel facts holds social value because it suggests that the sharer holds inside information^{Footnote 81}. The tendency to attend to negative information is well established. People attend more to potential losses than to gains^{Footnote 82}. Sharing negative information has the veneer of nobility by warning others of potential threats. Health professionals have been found to be more willing to retransmit false rumours to prevent negative repercussions (e.g., causing cancer) than to produce positive outcomes (e.g., curing cancer)^{Footnote 83}^{Footnote 84}.

People share with their ideological community because it satisfies a fundamental human motivation to strengthen one’s social attachments^{Footnote 85}. It also confirms one’s identity as part of an ideological group^{Footnote 86}. Protecting self-identity takes priority over judging accuracy^{Footnote 87}. Indeed, identity (e.g., nationality, religion, race, and/or political party) will colour what people consider to be true^{Footnote 88}. However, because people tend to live in information bubbles, partisan belief differences generally demonstrate an ignorance of inconvenient truths rather than an acceptance of falsehoods. Therefore, people may see and share a deepfake video that aligns with their beliefs while never coming across any reason to believe that it is in fact a deepfake. In one study showing subjects faked photographs, conservatives were more likely to “remember” Barack Obama shaking hands with the president of Iran, while liberals were more likely to “remember” George W. Bush on vacation with a celebrity during Hurricane Katrina (neither event actually happened)^{Footnote 89}. Deepfakes have been shown to radicalize people against the opposition^{Footnote 90}, therefore, like other information sources, deepfakes may be more likely to radicalize existing views than to change people’s opinions.

Older people are more likely to be deceived by deepfakes, and political ideology influences how deepfaked news is evaluated^{Footnote 91}. While Republicans and Democrats in the US are equally inclined to share fake news^{Footnote 92}, low-conscientiousness conservatives (e.g., those least likely to follow societal norms for impulse control) are the most likely to share misinformation due to their desire for chaos^{Footnote 93}.

Consequences

Societies, companies, and consumers are all potentially threatened by deepfakes. Caldwell et al. (2020) ranks fake audio or video content as the single biggest threat posed by AI for applications to crime and terrorism^{Footnote 94}. Europol (2022) has warned that deepfakes can be used to harass and humiliate people online, perpetrate extortion and fraud, falsify online identities and fool “know your customer” mechanisms, sexually exploit children online, falsify or manipulate electronic evidence for criminal justice investigations, and disrupt financial markets^{Footnote 95}.

Deepfakes also pose a threat to our governing structures. The uncertainty deepfakes introduce allows people to live in their own subjective realities, enlarging social divisions and obstructing the democratic process^{Footnote 96}. This is especially dangerous during elections when deepfakes are likely to be used by both foreign and domestic powers to manipulate outcomes^{Footnote 97}. Antagonistic parties may be enticed to subject an electorate to deepfakes long before an election in order to prime future attitudes^{Footnote 98}.

In regards to businesses, threats include fake reviews of consumer items, defamation and sabotage, and damage to a firm’s image, reputation, and trustworthiness^{Footnote 99}. For instance, in 2019 criminals successfully impersonated the head of a firm’s parent company with voice spoofing software thereby tricking the CEO of a UK energy company into transferring $243,000 USD to them.

Beyond such “deepfake phishing”, AI will render some technologies obsolete. Threats to consumers include new susceptibility to blackmail, intimidation, sabotage, harassment, defamation, revenge porn, identity theft, and bullying.

The personalized nature of deepfake pornography adds a new layer of emotional distress and threat for victims^{Footnote 100}. Most pornographic deepfakes present celebrities whose reputations may provide a degree of shelter from being seen as the genuine subjects of the videos. They also possess public platforms, as well as legal and financial means to dispute the veracity of the videos. In cases of revenge porn, private citizens do not have even these limited protections^{Footnote 101}.

Even when citizens do not believe the misinformation presented to them or are not concerned about truth, deepfakes can increase uncertainty about content and decrease trust in media^{Footnote 102}. In the US, fake news caused 50 per cent of Republicans and 38 per cent of Democrats to reduce the amount of news they consume^{Footnote 103}. As COVID-19 exemplified, in times of crisis this atmosphere of conspiracy and uncertainty can leave citizens vulnerable to misinformation^{Footnote 104}. Deepfakes are exacerbating this problem.

The use of deepfakes against public individuals creates the Liar’s Dividend: individuals facing accusations can write off factual evidence as deepfakes^{Footnote 105}. Widespread deepfakes can prime individuals into doubting the authenticity of information. The Malaysian Minister of Economic Affairs deflected evidence of his involvement in a sex tryst by proclaiming it as a deepfake despite no evidence^{Footnote 106}. More recently, Elon Musk’s lawyers used it in a lawsuit^{Footnote 107}.

Deepfakes offer plenty of potential benefits. They have enabled new and intriguing art forms, served as excellent pedagogical tools, and been a benign source of pleasure and amusement. They can also offer business opportunities^{Footnote 108}. Facebook’s metaverse will be largely composed of deepfake objects. Deepfakes afford new forms of marketing campaigns (e.g., through the removal of language barriers), virtual brand ambassadors (Lil Miquela is a fake influencer who has over 3 million followers), and a range of technical innovations. To illustrate, there are now virtual newsreaders based on real people. Deepfakes can also be used to enhance memory by, for example, making a dead person seem alive.

In the wrong hands, deepfakes are a novel kind of social virus, and like all viruses, their future trajectory and consequences are hard to predict. On a societal level, their greatest threat is their ability to shape public discourse. When misinformation enters the public conversation, it becomes increasingly dangerous as it alters collective understanding and memory. Their increasing prevalence could also lead people to stop believing much of what they see.

Solutions

Deepfakes are created to trick us; the human mind is not prepared to always accurately identify the outputs of sophisticated technologies. While some tech giants have started flagging some content as misinformation, such flags are not a silver bullet. The shareability of fake news has been found to decrease when it is accompanied by warnings^{Footnote 109}; however, their effect on its believability is unclear^{Footnote 110}. Prior exposure to misinformation increases its perceived accuracy, possibly negating the effectiveness of tags. Detection systems integrating both human and model predictions have been found to be more accurate than humans and automatic detection methods working alone^{Footnote 111}.

Steps that might alleviate the problem include pre-exposure warnings that make people aware that information might be false before they see it. Warnings need to be specific; it is ineffective to merely mention that misinformation may be present^{Footnote 112}. In addition, warnings should come with an alternative causal account that explains both what happened and the reason for the misinformation. Companies can educate consumers about their products, brands, and services, helping them identify firm-sponsored and credible sources of information^{Footnote 113}.

From a legal standpoint, there is currently little distributor liability for social media platforms circulating deepfakes^{Footnote 114}. In the United States, the legal debate is centered around Section 230 of the Communications Decency Act, which prevents companies from being held liable for the content on their platforms^{Footnote 115}. The justice system could specify civil liability for the creators and distributors of deepfakes, while also increasing legal protection for victims of defamation.

Individuals have little power to prevent deepfake attacks. When deepfakes threaten reputations, individuals can increase their ability to deny actions by recording their activities, but this raises privacy concerns^{Footnote 116}. Methods to disseminate facts can help protect communities if they are deeply informed by understanding of the public's information landscape and means of navigating it.

Individuals are more easily persuaded and corrected by someone they know. Therefore, societal norms and discourse on deepfakes should be nudged to create a social environment where people are not only more skeptical about what they see, but also are encouraged to challenge each others’ informational claims.

To alter societal norms, thought leaders and those most central in social networks are key. Educational resources including digital literacy training are helpful tools, especially if directed at influencers. Videos explaining political deepfakes have been found to reduce uncertainty, and in so doing can increase trust in media^{Footnote 117}. But norms only really change through collective action.

Footnote 61

This chapter was derived from: Nieweglowska, M., Stellato, C., & Sloman, S. A. (2023). Deepfakes: Vehicles for Radicalization, Not Persuasion. Current Directions in Psychological Science, 09637214231161321.

Return to footnote61 Referrer

Footnote 62

Ajder, H., Patrini, G., Cavalli, F., & Cullen, L. (2019, September). The State of Deepfakes: Landscape, Threats, and Impact.

Return to footnote62 Referrer

Footnote 63

Vaccari, C., & Chadwick, A. (2020). Deepfakes and disinformation: Exploring the impact of synthetic political video on deception, uncertainty, and trust in news. Social Media+ Society, 6(1), 2056305120903408.

Return to footnote63 Referrer

Footnote 64

https://www.npr.org/2022/03/16/1087062648/deepfake-video-zelenskyy-experts-war-manipulation-ukraine-russia.

Return to footnote64 Referrer

Footnote 65

https://www.buzzfeed.com/craigsilverman/obama-jordan-peele-deepfake-video-debunk-buzzfeed

Return to footnote65 Referrer

Footnote 66

Prior, M. (2013). Visual Political Knowledge: A Different Road to Competence? The Journal of Politics, 76(1), 41–57.

Return to footnote66 Referrer

Footnote 67

Messaris, P. &, Limus, A. 2001. The Role of Images in Framing News Stories. In Framing Public Life: Perspectives on Media and Our Understanding of the Social World, edited by Reese, Stephen D., Gandy, Oscar H., Grant, August E., 215–26. Mahwah: Lawrence Erlbaum.

Return to footnote67 Referrer

Footnote 68

Rim, S., Amit, E., Fujita, K., Trope, Y., Halbeisen, G., & Algom, D. (2015). How words transcend and pictures immerse: On the association between medium and level of construal. Social Psychological and Personality Science, 6(2), 123-130.

Return to footnote68 Referrer

Footnote 69

Kirkpatrick, A. (2020). The spread of fake science: Lexical concreteness, proximity, misinformation sharing, and the moderating role of subjective knowledge. Public Understanding of Science 30 (1), 55 - 74.

Return to footnote69 Referrer

Footnote 70

Barari, S., Lucas, C., & Munger, K. (2021). Political deepfake videos misinform the public, but no more than other fake media. OSF Preprints.

Return to footnote70 Referrer

Footnote 71

Return to footnote71 Referrer

Footnote 72

Return to footnote72 Referrer

Footnote 73

Lago F., Pasquini C., Böhme R. , Dumont H., Goffaux V. and Boato G., More Real Than Real: A Study on Human Visual Perception of Synthetic Faces [Applications Corner]. IEEE Signal Processing Magazine, vol. 39, no. 1, pp. 109-116, Jan. 2022.

Return to footnote73 Referrer

Footnote 74

Mori, M., K. F. MacDorman and N. Kageki. (2012). “The Uncanny Valley”. IEEE Robotics and Automation Magazine, 19(2), 98–100.

Return to footnote74 Referrer

Footnote 75

Köbis, N. C., Doležalová, B., & Soraperra, I. (2021). Fooled twice: People cannot detect deepfakes but think they can. IScience, vol. 24 (11), 1–17.

Return to footnote75 Referrer

Footnote 76

Lee, J., & Shin, S. Y. (2022). Something that they never said: multimodal disinformation and source vividness in understanding the power of AI-enabled deepfake news. Media Psychology, 25(4), 531-546.

Return to footnote76 Referrer

Footnote 77

Pennycook, G., Epstein, Z., Mosleh, M. et al. (2021). Shifting attention to accuracy can reduce misinformation online. Nature vol. 592, issue 7855 590–595.

Return to footnote77 Referrer

Footnote 78

Vosoughi, S., Roy, D., & Aral, S. (2018). The spread of true and false news online. Science, 359(6380), 1146-1151.

Return to footnote78 Referrer

Footnote 79

Ajder, H., Patrini, G., Cavalli, F., & Cullen, L. (2019, September). The State of Deepfakes: Landscape, Threats, and Impact.

Return to footnote79 Referrer

Footnote 80

Chesney, R., and Citron, D.K. (2019). Deep fakes: a looming challenge for privacy, democracy, and national security. Calif. L. Rev. vol. 107, issue 6, 1753–1820.

Return to footnote80 Referrer

Footnote 81

Groh, M., Epstein, Z., Obradovich, N., Cebrian, M., & Rahwan, I. (2021). Human detection of machine-manipulated media. Commun. ACM 64, 10 (October), 40–47.

Return to footnote81 Referrer

Footnote 82

Yechiam, E., & Hochman, G. (2014). Loss attention in a dual task setting. Psychological Science, vol. 25, issue 2, 494-502.

Return to footnote82 Referrer

Footnote 83

Return to footnote83 Referrer

Footnote 84

Bebbington, Jan & Russell, Shona & Thomson, I. (2017). Accounting and sustainable development: Reflections and propositions. Critical Perspectives on Accounting. 48. 10.1016/j.cpa.2017.06.002.

Return to footnote84 Referrer

Footnote 85

Baumeister, R. F., & Leary, M. R. (1995). The need to belong: Desire for interpersonal attachments as a fundamental human motivation. Psychological Bulletin, 117(3), 497-529.

Return to footnote85 Referrer

Footnote 86

Tajfel, H. (1974). Social identity and intergroup behaviour. Social science information, 13(2), 65-93.

Return to footnote86 Referrer

Footnote 87

Zhou, Y., & Shen, L. (2021). Confirmation Bias and the Persistence of Misinformation on Climate Change. Communication Research.

Return to footnote87 Referrer

Footnote 88

Liv., N & Greenbaum, D. (2020). Deep Fakes and Memory Malleability: False Memories in the Service of Fake News, AJOB Neuroscience, 11:2, 96-104.

Return to footnote88 Referrer

Footnote 89

Frenda, S. J., Knowles, E. D., Saletan, W., & Loftus, E. F. (2013). False memories of fabricated political events. Journal of Experimental Social Psychology, 49(2), 280-286. doi:10.1016/j.jesp.2012.10.013.

Return to footnote89 Referrer

Footnote 90

Dobber, T., Metoui, N., Trilling, D., Helberger, N., & de Vreese, C. (2021). Do (Microtargeted) Deepfakes Have Real Effects on Political Attitudes? The International Journal of Press/Politics, 26(1), 69–91.

Return to footnote90 Referrer

Footnote 91

Caramancion, K. M. (2021, April). The demographic profile most at risk of being disinformed. In 2021 IEEE International IOT, Electronics and Mechatronics Conference (IEMTRONICS) (pp. 1-7). IEEE

Return to footnote91 Referrer

Footnote 92

Harper, C. A., & Baguley, T. (2019). “You are fake news”: Ideological (a)symmetries in perceptions of media legitimacy. OSF preprints.

Return to footnote92 Referrer

Footnote 93

Lawson, M. A., & Kakkar, H. (2022). Of pandemics, politics, and personality: The role of conscientiousness and political ideology in the sharing of fake news. Journal of Experimental Psychology: General, 151(5), 1154.

Return to footnote93 Referrer

Footnote 94

Caldwell, M., Andrews, J. T. A., Tanay, T., & Griffin, L. D. (2020). AI-enabled future crime. Crime Science, 9(1), 14.

Return to footnote94 Referrer

Footnote 95

Europol. (2022). Facing reality? Law enforcement and the challenge of deepfakes, an observatory report from the Europol Innovation Lab, Publications Office of the European Union, Luxembourg. https://www.europol.europa.eu/publications-events/publications/facing-reality-law-enforcement-and-challenge-of-deepfakes.

Return to footnote95 Referrer

Footnote 96

Chesney, R., and Citron, D.K. (2019). Deep fakes: a looming challenge for privacy, democracy, and national security. Calif. L. Rev. vol. 107, issue 6, 1753–1820.

Return to footnote96 Referrer

Footnote 97

Liv., N & Greenbaum, D. (2020). Deep Fakes and Memory Malleability: False Memories in the Service of Fake News, AJOB Neuroscience, 11:2, 96-104.

Return to footnote97 Referrer

Footnote 98

Cialdini, R. B. (2018). Pre-suasion: A revolutionary way to influence and persuade. New York: Simon & Schuster Paperbacks.

Return to footnote98 Referrer

Footnote 99

Mustak, M., Salminen, J., Mäntymäki, M., Rahman, A., & Dwivedi, Y. K. (2023). Deepfakes: Deceptions, mitigations, and opportunities. Journal of Business Research, 154, 113368.

Return to footnote99 Referrer

Footnote 100

Harris, D. (2019, January 05). Deepfakes: False pornography is here and the law cannot protect You: Duke Law & Technology Review.

Return to footnote100 Referrer

Footnote 101

Gieseke, A. P. (2020). "The New Weapon of Choice": Law's Current Inability to Properly Address Deepfake Pornography. Vanderbilt Law Review, 73(5), 1479-1515.

Return to footnote101 Referrer

Footnote 102

Return to footnote102 Referrer

Footnote 103

Mitchell, Amy (2020, August 17). Many Americans SAY made-up news is a critical problem that needs to be fixed. Retrieved February 23, 2021, from https://www.journalism.org/2019/06/05/many-americans-say-made-up-news-is-a-critical-problem-that-needs-to-be-fixed/.

Return to footnote103 Referrer

Footnote 104

Ognyanova, K., Lazer, D., Robertson, R. E., & Wilson, C. (2020). Misinformation in action: Fake news exposure is linked to lower trust in media, higher trust in government when your side is in power. Harvard Kennedy School (HKS) Misinformation Review.

Return to footnote104 Referrer

Footnote 105

Chesney, R., and Citron, D.K. (2019). Deep fakes: a looming challenge for privacy, democracy, and national security. Calif. L. Rev. vol. 107, issue 6, 1753–1820.

Return to footnote105 Referrer

Footnote 106

Barari, S., Lucas, C., & Munger, K. (2021). Political deepfake videos misinform the public, but no more than other fake media. OSF Preprints.

Return to footnote106 Referrer

Footnote 107

https://www.npr.org/2023/05/08/1174132413/people-are-trying-to-claim-real-videos-are-deepfakes-the-courts-are-not-amused#:~:text=People%20are%20arguing%20in%20court%20that%20real%20images%20are%20deepfakes%20%3A%20NPR&text=Press-,People%20are%20arguing%20in%20court%20that%20real%20images%20are%20deepfakes,claim%20that%20anything%20is%20fake.

Return to footnote107 Referrer

Footnote 108

Mustak, M., Salminen, J., Mäntymäki, M., Rahman, A., & Dwivedi, Y. K. (2023). Deepfakes: Deceptions, mitigations, and opportunities. Journal of Business Research, 154, 113368.

Return to footnote108 Referrer

Footnote 109

Pennycook, G., Cannon, T. D., & Rand, D. G. (2018). Prior exposure increases perceived accuracy of fake news. Journal of Experimental Psychology. General, 147(12), 1865–1880.

Return to footnote109 Referrer

Footnote 110

Ternovski, J., Kalla, J., & Aronow, P. M. (2021). Deepfake Warnings for Political Videos Increase Disbelief but Do Not Improve Discernment: Evidence from Two Experiments. OSF preprints.

Return to footnote110 Referrer

Footnote 111

Groh, M., Epstein, Z., Obradovich, N., Cebrian, M., & Rahwan, I. (2021). Human detection of machine-manipulated media. Commun. ACM 64, 10 (October), 40–47.

Return to footnote111 Referrer

Footnote 112

Lewandowsky, S., Ecker, U. K., Seifert, C. M., Schwarz, N., & Cook, J. (2012). Misinformation and its correction: Continued influence and successful debiasing. Psychological Science in the Public Interest, 13(3), 106-131.

Return to footnote112 Referrer

Footnote 113

Mustak, M., Salminen, J., Mäntymäki, M., Rahman, A., & Dwivedi, Y. K. (2023). Deepfakes: Deceptions, mitigations, and opportunities. Journal of Business Research, 154, 113368.

Return to footnote113 Referrer

Footnote 114

Kietzmann, J., Lee, L. W., McCarthy, I. P., & Kietzmann, T. C. (2020). Deepfakes: Trick or treat? Business Horizons, 63(2), 135-146.

Return to footnote114 Referrer

Footnote 115

Chesney, R., and Citron, D.K. (2019). Deep fakes: a looming challenge for privacy, democracy, and national security. Calif. L. Rev. vol. 107, issue 6, 1753–1820.

Return to footnote115 Referrer

Footnote 116

Chesney, R., and Citron, D.K. (2019). Deep fakes: a looming challenge for privacy, democracy, and national security. Calif. L. Rev. vol. 107, issue 6, 1753–1820.

Return to footnote116 Referrer

Footnote 117

Return to footnote117 Referrer

Page details

Date modified:: 2023-11-16

Language selection

Search

Disinformation, Deepfakes, and the Human Response

Page details