🤥 Faked Up #11

Generative AI is brat | Spying the fact-checkers | DEFIANCE clears the Senate

Anything happen while I was out?

Faked Up #11 is brought to you by dumb generative AI and pristine couches. The newsletter is an ~8-minute read and contains 70 links.

---

Top Stories

GENERATIVE AI IS BRAT

Ugh. I guess I’m going to have to write about the Musk tweet. Quick recap for those who missed the action: Conservative creator Christopher Kohls (aka “Mr Reagan”) made a spoof Kamala Harris campaign ad featuring an AI voice clone of the US Vice President. Kohls added “parody” to both his X and YouTube posts, but Musk’s tweet contained no such disclosure.

Look, I don’t think anyone actually believed Harris called Biden the “ultimate Deep State puppet.” Not anyone who would be affected by a warning label, at least.

This surface-level deception is more meme than misinformation. It also appears to be the current norm for AI-powered political discourse globally. In elections from India to South Africa, generative AI is being used primarily to entertain hyperpartisan audiences rather than deceive undecided voters.

The Harris video suggests that much AI-generated election content in the US might end up being dumb rather than dangerous. It also exposes the brittleness of the AI detection and labeling infrastructure.

AI audio generators have promised to ban voice clones of elected officials; once again they have failed to do so. Tech platforms have promised to prominently label AI-generated content; once again they have failed to do so.

Because community notes on Musk’s tweet are a partisan food fight, no label has been added to his video. But set aside the social network whose CEO spent the past few days posting antivax memes and racist claims about importing voters.

Take YouTube instead. Google’s video platform promised in March that it would add a “prominent label” on videos that use generative AI and discuss “sensitive topics” like elections. In the mockup, the label appears as an overlay on the full-screen video.

Yet on the Harris deepfake, the disclosure is visible only if you click “more” in the video description. This is on a video whose creator was open about his use of synthetic audio and (I’m assuming) had voluntarily added this information to his video upon uploading it.

Again, I really don’t think American democracy was harmed by this video. But it is a reminder that (a) generative AI is dumb (b) voice clones are going to be commonplace and (c) labels for AI content are not ready for prime time.

PS. Think you can spot an audio deepfake? Try this quiz¹.

IT CAN HAPPEN TO ANYONE

Cybersecurity firm KnowBe4 disclosed that it mistakenly hired a North Korean IT worker who was masquerading as a real US-based engineer. In their words:

KnowBe4 needed a software engineer for our internal IT AI team. We posted the job, received resumes, conducted interviews, performed background checks, verified references, and hired the person. We sent them their Mac workstation, and the moment it was received, it immediately started to load malware.

The scammer used AI to turn a stock image into a unique headshot (see below). KnowBe4 ran four video interviews “confirming the individual matched the photo provided on their application,” so I’m assuming this was a face swapping tool rather than a generator.

BAD AT BREAKING

The University of Washington’s Center for an Informed Public framed the misinformation that emerged after the Trump shooting as an element of “collective sensemaking.” During moments of crisis, it is natural for rumors to circulate as most news is unverified and folks try to understand what’s happened as a group.

And good luck turning to AI for answers. Both NewsGuard and the Washington Post found that chatbots frequently refused to engage with news of the shooting, called it misinformation, or failed to debunk emerging falsehoods.

I found particularly interesting this analysis by Conspirador Norteño on X Community Notes debunking false claims about the attempted assassination. Only about 9% out of the 1000+ notes that the analyst reviewed were deemed helpful and ended up attached to a tweet. On average, it took 15 hours from submission to public display (see graph below). That’s not warp speed, but it isn’t terrible either.

Also notable is that “35 of the 83 posts with visible notes (42.2%) were subsequently deleted by their authors,” suggesting Notes have some level of shaming effect.

THE LONG TAIL

Approximately 20% of Americans surveyed by Morning Consult two days after the attempted assassination of Donald Trump said “they find it credible that the shooting was staged and not intended to kill.” This included 12 percent of Trump’s own supporters.

It’s too early to tell whether that or multiple other conspiracy theories will stick, but misinformation was top of mind for some at yesterday’s Senate hearing on the shooting. Acting Secret Service Director Ronald Rowe Jr. had this to say:

I regret that information was not passed to Congress and the public sooner and with greater frequency. I fear that this lack of information has given rise to multiple false and dangerous conspiracy theories about what took place that day. I want to debunk these conspiracies today by sharing the following confirmed details.

40 minutes later, Senator Amy Klobuchar did not seem convinced those details had been enough:

For the people in my state that keep asking me: I just don’t get how [the shooter] got on the roof — I know we’ve gone through great details and a lot of examination — could you just give a minute on what went wrong and how you think it can be fixed? Because I think it’s just going to help to dispel the conspiracy theories. There are some people that think it didn’t really happen, which of course is completely ridiculous, it did. There are some people that think all kinds of conspiracies went on within the government, which is also false.

SPYING THE FACT-CHECKERS

The Brazilian police has arrested five individuals tied to a clandestine spying ring run between 2019 and 2022 within ABIN, the country’s intelligence agency. The country’s Supreme Court released documents claiming that the criminal activity was aimed at obtaining politically advantageous information and producing disinformation dossiers to attack those perceived as rivals of the administration of Jair Bolsonaro. Among the targets of the illicit investigations were fact-checking organizations Aos Fatos and Lupa.