Finding Signals in the Synthetic: Intelligence in the Era of Deepfakes

That deepfake videos will likely have a negative effect on our information environment has been well established. Less considered, however is the extent to which deepfakes will impact intelligence and national security agencies, including the threat environment they operate in, intelligence collection methods, the use of automated threat detection processes, the reception of intelligence products, and ethical dilemmas.

Despite the impressive technological nature of deepfakes, they are more likely to evolve current national security and intelligence threats rather than generate new ones. Instead, a more serious array of challenges may arise from ethical dilemmas that will require excellent judgement amid a collection of difficult choices ahead.

Evolving Threats

Generally, much of the literature on deepfakes focuses on the threats that they pose to democratic societies. Importantly, most of these threats are not likely to be new, but evolved and enhanced versions of threat-related activities that intelligence and national security agencies are already dealing with. This includes disinformation, the targeting of government/military personnel by adversarial forces, phishing/social engineering, and mimicking biometric data.

Disinformation

For the purpose of this paper, disinformation is defined as “false information that is intended to manipulate, cause damage, or guide people, organizations, and countries in the wrong direction”. Similar to disinformation, malinformation is information that stems from the truth but is often exaggerated in a way that misleads and causes potential harm ^{Footnote 127}. Malinformation often stems from stolen or hacked information, some of which may be altered to lend credibility to a false narrative that an adversary wishes to emphasize, and then released on the internet for distribution.

Current online media ecosystems are awash with large amounts of disinformation and malinformation, which serve a number of different ends. From a national security perspective, this largely involves foreign interference and radicalization activities. It is well established that states such as Russia, China, and Iran engage in disinformation and malinformation campaigns to serve their political objectives. Likewise, violent extremist groups spread narratives about societal collapse, corrupt institutions, global conspiracy theories, and that violent, revolutionary action is required to restore humanity to its rightful condition (although this may be accomplished through the use of irony and memes) ^{Footnote 128}. Importantly, despite their different political objectives, what often unites these two groups is their efforts to discredit and downplay democratic institutions, amplify conspiracy theories, and encourage distrust of what they see, generally, as “the system”.

The advantage of deepfakes for these actors is that they lower the cost of engaging in disinformation campaigns. Whereas in the past it may have taken time, effort, and skill to generate forgeries, and false information, deepfakes will quickly generate materials that can be utilized quickly, and spread worldwide even faster. Depending on how widespread and accessible deepfake tools are, they may also allow individuals to participate in information wars, thereby muddying an already complex information environment.

Targeting Government and Military Personnel

Adversarial actors will likely use deepfakes to target government, military, and national security personnel, usually for the purposes of making them targets or disrupting their work.

Already, there are disinformation campaigns taking place where western troops are posted, in order to breed mistrust and poor relations with civilian populations. For example, as Canadian troops deployed to Latvia as part of NATO’s Enhanced Forward Presence, a disinformation and malinformation campaign targeting them appeared^{Footnote 129}. These efforts at sowing distrust have been ongoing since 2017^{Footnote 130}. It is possible that deepfakes will be used to aid in these disinformation campaigns. Alternatively, the families of military personnel could be targeted with deepfakes involving their loved ones who may be serving abroad, in order to cause anguish and mental harm.

Social Engineering

A third area of concern is threats related to phishing or social engineering. Social engineering is the practice of obtaining confidential information by manipulation of legitimate users. Typically, social engineers use the telephone or internet to trick people into revealing sensitive information by pretending to be a figure of authority, co-worker, family member or even tech-support. This includes phishing, where a malicious actor sends an email mimicking or spoofing a specific, usually well-known brand, to convince someone to provide confidential information^{Footnote 131}.

There is also concern that deepfakes, which can replicate the face, image, and voice of individuals, may trick people in more advanced ways. While this will almost certainly be a boon to criminals, adversarial intelligence agencies may use it to target politicians, intelligence officers or other holders of classified information in order to gain their trust and subsequent access to sensitive data. Moreover, in conflict, deepfakes may be used as ruses of war where fake videos or audio may be used to send false orders or commands to troops, or false information to disrupt military operations.

Hacking Biometric Data

Moving beyond social engineering to obtain classified or sensitive information, adversaries may use deepfakes to mimic biometric data in order to gain direct access. Research suggests that deepfakes may already have the capacity to fool biometric scanners, such as facial recognition systems^{Footnote 132}. Given that an increasing number of applications are collecting and using biometric data, it is likely that a significant number of institutions holding this data may either sell it or be susceptible to hacking^{Footnote 133}. Furthermore, this biometric data may be used to create deepfakes that are even more realistic.

Intelligence Collection

In responding to current, evolving, and future threats, national security and intelligence agencies are tasked with collecting information that pertains to their mandate. Unfortunately, it is likely that this too will be impacted by deepfakes in at least two ways: creating noise and targeting open-source information.

Creating Noise

Deepfakes may be employed as a disruption tool by adversarial states against intelligence collection. This could include signals intelligence should deepfakes be used to flood an information space, thereby creating lots of “noise” or false distraction. They may also be used tactically against a suspected, specific collection.

It is also possible that deepfakes could distort the perception of human sources who believe that an artificially generated conversation, video, or text is real, and subsequently pass that on to intelligence collectors in good faith. If a human source is unable to differentiate between true and fake information, it could impact intelligence collection and analysis.

Open-Source

A second issue relates to the use of open-source information by both government and non-government agencies. The 2022 invasion of Ukraine by Russia is the latest global event demonstrating the significance and value of open-source information and analysis^{Footnote 134}. Ranging from scraping social media through to analysis of publicly available satellite imagery, open-source techniques are being used to uncover troop movements, defensive fortifications, gain insight into the morale of combatants, verify attacks, losses, military strikes, and to investigate war crimes. Although the quality may vary, both national security agencies as well as journalists and humanitarian organizations have developed their own techniques or found reliable sources to inform their investigations. As such, open-source information is a prime target for deepfakes. Adversarial actors seeking to create division amongst allies, weaken resolve, deny war crimes, or falsify information will likely target open-source outlets with deepfakes. This may lead to incorrect reporting, which could then be used against open-source outlets to discredit their efforts. Even in a best-case scenario, deepfakes may make the already time-consuming job of open-source information verification much more difficult.

Automated Processes

It is also possible that deepfakes could impact automated processes designed to thwart adversarial activities. Data poisoning occurs when trawled data for deep-learning training of machine learning systems is compromised intentionally with malicious information^{Footnote 135}. Algorithms used to detect cyber-attacks, or disinformation/malinformation campaigns could also be compromised through data poisoning of the large-scale sets of information they are trained on. Moreover, researchers have found that systems designed to detect deepfakes can be affected by data poisoning, rendering them less effective^{Footnote 136}.

Reception of Information

As noted above, a key concern over deepfakes is the role they may play in worsening an already convoluted information environment. Therefore, while operating in an information space where the truth is increasingly contested, government officials, executives within national security and intelligence communities, and their analysts should anticipate challenges when it comes to having their findings accepted by the public or even politicians.

Intelligence assessments should always be questioned and/or interrogated by their audiences. However, where questioning is guided by accusations and challenges stemming from conspiracy theories, misinformation, disinformation and/or malinformation rather than the interests of good governance, the position of intelligence and national security agencies will be much more difficult. In particular, the social license that these agencies require to perform their jobs will be put at risk if a significant segment of the population rejects their findings outright, or ignores them because discerning the truth is seen as too difficult. This problem may be aggravated where these departments, and agencies have traditionally struggled with transparency.

Complicating matters further, warnings about deepfakes may actually reinforce the problem in some information ecosystems. Chesney and Citron note that efforts to warn the public about the pernicious effects of deepfakes may have a perverse outcome they call the “Liar’s Dividend”. In this scenario, individuals, corporations, and governments accused of engaging in harmful actions will be able to claim that any evidence produced, especially images, audio and/or videos are deepfakes, in an effort to dodge responsibility^{Footnote 137}.

Ethics of Deepfakes

Given the concerns addressed above, researchers and scholars, particularly from a legal, scientific, and/or technical perspective, have focused on finding technical and regulatory solutions. Few articles have explored the ethical dilemmas generated by deepfakes, particularly for government departments and agencies. This paper will briefly discuss three of these dilemmas: the use of deepfakes and democratic norms, private sector dilemmas, and the risk of “over-hyping” the issue.

Should democracies use deepfakes?

The first challenge is that if deepfake techniques prove to be inexpensive and effective, there will be temptation to use them in the defence, security and intelligence operations of democratic countries. On the one hand, these states may wish to use these techniques because they are cost effective, and may be easier than other, riskier forms of intelligence gathering or covert activities.

For agencies that wish to use deepfakes, it may be argued that ruses of war have existed for centuries. Furthermore, a key goal of present information operations is the dissemination of propaganda in pursuit of a competitive advantage over an opponent^{Footnote 138}. This includes attempts to induce a sense of helplessness in an adversarial military or population, so they do not wish to fight^{Footnote 139}. Therefore, it will not be surprising if states manufacture deepfakes as a part of these campaigns to achieve their goals quickly, easily, and potentially with minimal bloodshed. Similarly, many intelligence agencies engage in disruption operations to prevent malicious activities from occurring on their territory or against their interests. Deepfakes could be used to mislead or fool adversaries with fake audio and video.

There is, however, a serious trade-off in doing so. It is expected that authoritarian states actively engage in propaganda and are very likely to turn to deepfakes to further their political objectives. However, given that democracies are grounded in the rule of law (however imperfect), they will not necessarily benefit in the same way from engaging in disinformation (nor is it clear if they are particularly good at information operations^{Footnote 140}). If it is known or believed that democracies, their militaries, and intelligence agencies are actively using deepfakes, the Liar’s Dividend will certainly take effect in instances that will matter down the road, particularly if the West is trying to persuade new or skeptical audiences.

Moreover, as disinformation is widely recognized as a problem affecting democracies, it is questionable if creating more of it through deepfakes is a good idea. After all, western intelligence agencies seem to have had more luck with “pre-bunking” disinformation during the 2022 Russian invasion of Ukraine rather than creating an alternative set of lies^{Footnote 141}.

Private Sector Dilemmas

A second series of ethical challenges is related to the role of the private sector in the creation, and detection of deepfakes. While AI and deepfake tools may enable a large number of independent, and proxy actors to engage in disinformation campaigns, it is possible that the real beneficiaries will be a small number of large, high net worth technology companies. Companies that have the means to amass, harness and process large datasets into machine learning systems, which can be used to both create, and detect deepfake content. How states work with these companies and use their products will require special care and consideration. Many machine learning datasets are based on images obtained through questionable means^{Footnote 142}. Concerns have been raised about racial bias in AI that can exacerbate systemic racism, and scientists have demonstrated that deepfake images can exacerbate racial bias in web-based face recognition APIs^{Footnote 143}. Deepfake algorithms may contain hidden racial and other biases that will affect outcomes.

Laws and privacy regulations will provide some guidance on what democratic states will be allowed to do. However, ethical judgement over what kinds of companies that states wish to engage with, how their practices are reviewed, and how to manage issues of accountability will be required.

Is the threat over-hyped?

Finally, for all the challenges that have been discussed in this paper, there is also a risk of exaggerating the threat. Disinformation is a serious problem, and even preliminary deepfakes may be contributing to it. However, many claims about the potential disruptive impact of AI-enabled propaganda are speculative and largely unscrutinised^{Footnote 144}. While AI will pose challenges, the present hype is not reality—deepfakes may be technically impressive, but this does not necessarily make their use practical. For example, a deepfake video of a world leader declaring war can quickly be checked and debunked simply by examining events on the ground.

Additionally, it is not immediately obvious that deepfake propaganda will be any more effective at sharing narratives than crudely-made images and memes, which are already widely and rapidly shared. Research has shown that fake news content spreads not because it is logical or realistic, but because it resonates emotionally with the sharer^{Footnote 145}. In this sense, states should be more concerned about certain narratives, rather than how good the content looks. States need to take deepfakes seriously—but in many cases they are evolving the current threat environment—not upending it. Therefore, overreaction to deepfakes may distort threat analysis and policy responses.

Conclusion

The above identifies some of the challenges (and opportunities) that national security, and intelligence agencies will face in the coming years. In doing so, it is argued that although the technology is impressive, deepfakes are more likely to evolve already existing threat-related activities, rather than generating new ones. If there is a silver lining to the deepfake dark cloud, it is that most democratic states are not starting from scratch but rather already have policies and procedures in place to help them manage deepfakes—although these too will need to evolve. For example, when collecting digital media, it will be important to establish chains of custody to help preserve and verify in the future or in law enforcement proceedings.

Many of the most challenging deepfake problems will not be solved with technology or law, but through ethical practices that will require good judgement. This includes thinking about how states should engage with the private sector, particularly those companies that already control large technology platforms, and what this means for oversight and review. Additionally, while there may be good reasons for democracies to consider the use of deepfakes for their own national security and intelligence operations, there may be more pitfalls than promise with this approach.

Footnote 127

Canadian Centre for Cyber Security, “How to identify misinformation, disinformation, and malinformation (ITSAP.00.300)”, February 2022. https://www.cyber.gc.ca/en/guidance/how-identify-misinformation-disinformation-and-malinformation-itsap00300.

Return to footnote127 referrer

Footnote 128

See, for example, Whitney Phillips and Ryan M. Milner, You Are Here: A Field Guide for Navigating Polarized Speech, Conspiracy Theories and Our Polluted Media Landscape, Cambridge, MA: MIT Press, 2021.

Return to footnote128 referrer

Footnote 129

Tom Blackwell, “Russian fake-news campaign against Canadian troops in Latvia includes propaganda about litter, luxury apartments”, National Post, 17 November 2017. https://nationalpost.com/news/canada/russian-fake-news-campaign-against-canadian-troops-in-latvia-includes-propaganda-about-litter-luxury-apartments.

Return to footnote129 referrer

Footnote 130

Murray Brewster, “Canadian-led NATO battlegroup in Latvia targeted by pandemic disinformation campaign”, 25 May 2020. https://www.cbc.ca/news/politics/nato-latvia-battle-group-pandemic-covid-coronavirus-disinformation-russia-1.5581248.

Return to footnote130 referrer

Footnote 131

Canadian Centre for Cyber Security, “Glossary”, Accessed 12 May 2023. https://www.cyber.gc.ca/en/glossary.

Return to footnote131 referrer

Footnote 132

Hannah Smith, and Katherine Mansted, Weaponised deep fakes; National security and democracy. Canberra: Australian Strategic Policy Institute, 2020.

Return to footnote132 referrer

Footnote 133

Ash Carter and Laura Manley, Tech Factsheets for Policymakers: Deepfakes, Cambridge, MA: President and Fellows of Harvard College, 2020; Office of the Privacy Commissioner, “Data at Your Fingertips Biometrics and the Challenges to Privacy”, February 2011. Also consider authoritarian states are collecting and centralizing biometric data as a part of their domestic surveillance programs. Riddle Russia, “Biometrics as a Kremlin tool”, 5 May 2023. https://ridl.io/biometrics-as-a-kremlin-tool/.

Return to footnote133 referrer

Footnote 134

Economist, “Open-source intelligence is piercing the fog of war in Ukraine”, 13 January 2023. https://www.economist.com/interactive/international/2023/01/13/open-source-intelligence-is-piercing-the-fog-of-war-in-ukraine.

Return to footnote134 referrer

Footnote 135

Payal Dhar, “Protecting AI Models from “Data Poisoning” New ways to thwart backdoor control of deep learning systems”, IEEE Spectrum, 24 March 2023 https://spectrum.ieee.org/ai-cybersecurity-data-poisoning.

Return to footnote135 referrer

Footnote 136

Xiaoyu Cao, Neil Zhenqiang Gong, “Understanding the Security of Deepfake Detection”, in Pavel Gladyshev, et al. (eds) Digital Forensics and Cyber Crime. ICDF2C 2021. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, Vol. 441. Springer, Cham, 2022. https://doi.org/10.1007/978-3-031-06365-7_22.

Return to footnote136 referrer

Footnote 137

Chesney, R. and Citron, D.K. (2019). Deep Fakes: A Looming Challenge for Privacy, Democracy, and National Security. SSRN Electronic Journal. doi:10.2139/ssrn.3213954.

Return to footnote137 referrer

Footnote 138

RAND Corporation, “Information Operations”. https://www.rand.org/topics/information-operations.html Accessed 17 May 2023.

Return to footnote138 referrer

Footnote 139

Clint Watts, Messing with the Enemy: Surviving in a Social Media World of Hackers, Terrorists, Russians and Fake News, New York: Harper Collins Publishers, 2018.

Return to footnote139 referrer

Footnote 140

Brett Boudreau, “The Rise and Fall of Military Strategic Communications at National Defence 2015-2021: A Cautionary Tale for Canada and NATO, and a Roadmap for Reform”, Canadian Global Affairs Institute, May 2022. https://www.cgai.ca/the_rise_and_fall_of_military_strategic_communications_at_national_defence_2015_2021

Return to footnote140 referrer

Footnote 141

Stephanie Carvin, “Deterrence, Disruption and Declassification: Intelligence in the Ukraine Conflict”, CIGI Online, 2 May 2022. https://www.cigionline.org/articles/deterrence-disruption-and-declassification-intelligence-in-the-ukraine-conflict/ ; David Klepper, ‘”Pre-bunking’ shows promise in fight against misinformation”, Associated Press, 24 August 2022. https://apnews.com/article/technology-misinformation-eastern-europe-902f436e3a6507e8b2a223e09a22e969.

Return to footnote141 referrer

Footnote 142

Chloe Xiang, “AI Is Probably Using Your Images and It's Not Easy to Opt Out”, Vice, 26 September 2022. https://www.vice.com/en/article/3ad58k/ai-is-probably-using-your-images-and-its-not-easy-to-opt-out.

Return to footnote142 referrer

Footnote 143

Shahroz Tariq, Sowon Jeon and Simon S. Woo, “Evaluating Trustworthiness and Racial Bias in Face Recognition APIs Using Deepfakes”, Computer, Vol. 56, No. 5, May 2023. doi: 10.1109/MC.2023.3234978.

Return to footnote143 referrer

Footnote 144

James R. Ostrowski, “Shallowfakes: The danger of exaggerating the AI disinfo threat”, The New Atlantis, Spring 2023. https://www.thenewatlantis.com/publications/shallowfakes.

Return to footnote144 referrer

Footnote 145

Cameron Martel, Gordon Pennycook and David G. Rand, “Reliance on emotion promotes belief in fake news”, Cognitive Research: Principles and Implications, Vol. 5, Article 47, 2020. https://cognitiveresearchjournal.springeropen.com/articles/10.1186/s41235-020-00252-3.

Return to footnote145 referrer

Page details

2026-09-03

Language selection

Search

Finding Signals in the Synthetic: Intelligence in the Era of Deepfakes

Page details