Interview with Renée DiResta
Maarten Muns
There already have been many elections this year. In the EU, but also in important countries like India, Indonesia and Mexico. In the run up to these elections, there was some fear of a massive amount of disinformation, especially fueled generative AI tools. The presidential elections in the US are the next major political event. But things fell a bit silent on the issue. What is your take on that?
“Every single election since 2016 has seen a lot of concern about deepfakes. There is a risk, but sometimes discussion of that risk is overblown. There are a couple of things to understand. One, there is an increasing democratization of access to AI and deepfake tools. Two, the tools have gotten better and what they produce is harder to detect. Three, they create some interesting challenges, particularly because as people become aware that high-quality generative AI content exists, it can also be used to cast doubt on real audio or video. So if a real audio fragment of a politician saying something scandalous comes out, their followers could say that it’s fake, that it’s not really them.”
“So there are interesting shifts that generative AI has made possible with regard to our ability to understanding what’s real. But I think that there are also a couple of other factors in play that limit its effectiveness. One, to be impactful, the generated content has to get wide distribution. Two, it has to be sufficiently high quality that people cannot quickly deduce what it is, so they believe it. And finally, maybe most importantly, there is the issue of trust: do people trust the fact checkers when they put out the authentication? In some places, like the United States, this may be very split along partisan lines.”
Do you think that the use of genAI tools is still a threat to fair elections?
“It is a new tactic that changes what is possible in the realm of manipulation. We’ve seen in the US that President Trump accused Kamala Harris of faking her crowds and said that some images of her on the tarmac are AI. And then the news media point out the original. We have seen Trump tweeting AI generated images of people wearing ‘Swifties for Trump’ T-shirts, of which some appear to be real and others appear to be AI generated. So one candidate has clearly picked up these tools and inspired questions about ‘what is real? Is this image an authentic representation of reality?’ Generative AI is becoming a low-key part of the political discourse here in the US. But it’s not something big or sensational, and it hasn’t yet been in used in what we sometimes call the October surprise, which is a big reveal that shakes up the race. We need to understand and communicate the risks to the public, but we can convey that information responsibly without saying that the sky is falling. So, yes, it can be potentially quite impactful. There are about two months until the election and a lot can change. But I think we need to “panic responsibly”, as elections expert Katie Harbath puts it.”
You wrote a lot about the spread of conspiracy theories in your latest book, Invisible Rulers. What is their role in the political landscape?
“Elections are a pivotal moment. Everyone is paying attention because there’s a major event that is about to happen that potentially changes the direction of the country. In the United States, we have longstanding and prominent conspiracy theory that the 2020 election was stolen. That, in my opinion, is the conspiracy theory that matters most in the political landscape. Conspiracy theory communities are persistent, they may activate around an election but often are not tied to a specific event, and they are marked by an intense and persistent distrust. They often do not believe what fact checkers or mainstream media say. In the conspiracy theories about the 2020 election, no amount of fact-finding or investigations have swayed their belief that President Biden was not legitimately elected. Today they have more potential for visibility than ever before, as they are well-networked on the communication channels that matter.”
“There is also a counter-cultural appeal in the U.S. right now, particularly post-COVID, where it is common among some political factions to say things like ‘conspiracy theories are reality that just hasn’t been exposed yet.’ Some of the most prominent voices on platforms like X use this kind of reflexive contrarianism and the mystique and appeal of alternative versions of reality to present themselves as free-thinkers, and to build an audience around that. It’s an appealing identity for some, and they are more prominent in the political conversation as a result. However, this is independent of generative AI. The harms of generative AI are not primarily political, nor conspiratorial – nonpartisan, non-election-related issues are far more important.”
Like what?
“I would argue that number one is non-consensual intimate imagery. AI generated images of women made without their consent are being released and published on the internet, and the task of explaining: ‘No, that’s not me’ suddenly falls on the victim. In the context of children, people are generating photorealistic child exploitation content. That is horrible. And it’s unclear how the law will treat these cases in the United States. There are loopholes in existing laws around revenge porn that need to be addressed and dealt with immediately. These are areas of harm where generative AI is transformative at a societal level.”
“Another area where you’re seeing a lot of AI abuse is in scams. People are making images of products that don’t exist to take money from others. They’re using AI tools to chat with targets, to get them to give up personal information. Generative audio is now being used in scams as well. You get a call from somebody you think is your boss to wire money, to make a trade, to log in to an email, or to give up your password. The focus on political manipulation captures a lot of attention. But generative AI is having a much more immediate impact on less-noticed things, such spam, scams, and exploitation content.”
What is the best way that governments can protect its citizens from these kinds of actions of harassment or manipulation?
“I think that the efforts to explain to people how generative AI works and what is makes possible are very important, particularly in the United States where regulation is going to be extremely challenging to pass. But leaders must to do this in a way that doesn’t create paranoia. One of the things that I noticed happening here in the US as the news of the Russian interference operation in 2016 emerged, was that for a period of time, suspicious people on the left who would encounter a MAGA account on Twitter, would say: ‘Oh, you’re a Russian bot, right?’ They had all heard about Russian bots in the news and they were over-reacting, thinking these bots were everywhere. This is very interesting because, as bots have become more interesting again through LLM (Large Language Model) powered bots, there will once again be more bots. People will in fact engage with bots inadvertently at times, but you need to balance that awareness with minimizing paranoia about it. Yes, there are AI generated accounts in the world, butat this point in time you are likely still engaging with a real person.”
“Public information campaigns can offer some ways to help people make the determination. That said, this is not a solution and it is challenging because AI generative content systems evolve very quickly. I remember I used to get asked about this in the days of generative adversarial network-produced fake faces, before we got to diffusion model generated images. We would tell people to look at the ears, because the earrings are often wrong or the hair blends into the collar. Look for these details, because the models consistently made mistakes. When we moved into generative images, people would say to look at the fingers. They often generate extra fingers. So you try to teach people, here’s the thing you can look for. The problem is that the models no longer make these kinds of mistakes. They’ve evolved past that.”
“So, if you tell somebody, look at the fingers and then the fingers look normal, you’ve inadvertently made them come to the wrong conclusion. Perhaps the plant in the corner blends into the wall, but you didn’t tell them that, so they didn’t look at that. Or the stools have four legs, but they’re not even. We need an informed public, but you don’t want people to go through life with such an intense degree of intense skepticism where they feel obligated to scrutinize every image. But at the same time, we do want them to be aware that this type of manipulation can and does occur, and that not all images are trustworthy. It’s frankly very difficult. I don’t have a really easy answer right now.”
How do our adversaries like China and Russia try to use these tools, for instance to push certain narratives? How can we act on that
“A couple of things are happening. First, some states are developing their own models. They have the technological capacity to do that. They are also, it seems, using OpenAI’s tools. OpenAI has begun releasing reports on adversarial use of its interfaces of ChatGPT. In response, there are threat detection teams within some of the bigger platforms that are trying to find state actors and disrupt them and, most importantly, transparently report on what these agents are doing. OpenAI recently reported on one related to Iran. Iranian actors were using OpenAI’s platforms to create content farms. To produce plausible content, you need it to be fluent in the language and slang of the community you’re targeting. And oftentimes they don’t get that quite right. For example, they mix up metaphors. The Russians rather notoriously pretended to be Black Americans in order to target Black communities, but didn’t use African American Vernacular English properly.”
Do you think a country like Russia will try to interfere in the US elections again this year?
“There’s no reason why they wouldn’t try. But there are interesting shifts, compared to 2016 for instance; our DOJ has just indicted some individuals linked to RT for paying right-wing influencers to produce propaganda. The tech platforms are actively looking for Russian interference at all times. There are teams that are focused entirely on that, and they are constantly taking down the new accounts. The government does investigations too. It’s looking at changes or evolution in established threat actor spaces, and communicating that information to the tech platforms. Facebook is constantly publishing threat indicators so that outside security researchers and other platforms can also look. Putting out threat information, treating it almost like a cybersecurity problem with persistent malign actors that evolve tactically over time, is how the tech community, the researcher community and even the government have evolved to thinking about this.”
“So, will they try to do something? Yes, obviously. Will it be impactful? Well, honestly, they have not had much success yet. And as I mentioned before, they need trusted accounts for doing that distribution. Those take a lot of time to develop. So I think it would be more likely for Russia to perhaps try to pay real influencers to spread their content at this point, as opposed to creating fake accounts on social media, growing influential and being a fake persona. I think that shift is very real.”
Photo: Shutterstock.com / Chaosamran_Studio
