🤥 Faked Up #22

Hurricane season misinfo is worse than usual, TikTok served me 59 conspiracy videos in a session of 100, and a fake Justin Bieber clip does the rounds again.

This newsletter is a ~6 minute read and includes 53 links.

---

HOW AM I DOING?!

👋 Hello! I have been sending Faked Up for almost 6 months and I’d like to get a sense of how it’s landing.

There’s only so much I can glean by (obsessively) reviewing open and click rates, so I’m asking you to answer five quick multiple choice questions (plus an open-ended one if you’re feeling chatty).

Thank you! 🙏 🙏 🙏 🙏

---

HEADLINES

WhatsApp wants to add a “Search the web” button to fight misinformation. South Korea is one signature away from criminalizing the creation and consumption of synthetic non consensual intimate imagery. Millions are using Telegram’s nudify bots. Italian Prime Minister Giorgia Meloni testified in a case against the creator of a deepfake porn featuring her likeness. Meta thinks mundane fakes of the Northern Lights are a good use case of its AI image editor; anti-Muslim propaganda is a more likely use case. Reality Defender is pitching a real-time deepfake video detector. OpenAI terminated naughty state actor accounts.

---

TOP STORIES

STORMS OF MISINFORMATION

Hurricanes spawn misinformation. They are frightful events with dramatic images that are hard to independently verify. Communities engage in rumor-laden collective sensemaking. There’s a reason Highway Shark is a misinfo meme.

Even so, this Atlantic hurricane season feels different.

Conspiracy theorists claimed Helene and Milton were geo-engineered to grab minerals or eliminate Republican voters.

The same folks who reject the scientific consensus that says human actions on aggregate cause climate change are trying to mainstream the idea that individual humans can literally create and direct extreme weather events.

They may be breaking through: NewsGuard claims that in the two weeks to Oct. 9, “174,000 articles and social media posts mentioned the hurricanes alongside terms such as ‘geoengineered,’ ‘manipulated,’ and ‘weather weapons.’”. Even if we generously assume that half of those are rebuttals to the conspiracies, that’s a 15,000% increase on the conversation about this topic in the two weeks prior.

True believers are taking a victory lap. In a recent YouTube video, Dane Wigington claimed “the awakening to the ongoing climate engineering operations taking place over our heads is now accelerating by the day.” Incidentally, Google is still hosting “The Dimming,” Wigington’s debunked documentary, and running ads for it.

Somehow these man-made targeted hurricanes were also not destructive enough. Here’s Wigington again: “it seems that the weather-makers decided to change the weather script. Was Milton’s impact anywhere near the level that Matrix media sensationalized all week long?” (“Milton overhyped” was the second suggestion I got on TikTok when searching for the Hurricane earlier this week.)

As Kate Starbird put it on Lawfare: “Just like the heat of Gulf of Mexico has supercharged the storms, the dynamics of our information platforms are supercharging the rumoring and conspiracy-theorizing that’s happening around these events.”

These dynamics now include a functionally useless Twitter.

During Hurricane Irma, I evacuated St Pete and monitored the platform for updates to share with my friends still in Florida. At the time, it was a great place for a moderately savvy user to track live events; now, there is no signal among the noise. As John Herrman writes, the service Twitter provided in the mid-2010s “was one of those small golden eras you didn’t realize you were in.”

IRL OR IT DIDN’T HAPPEN

Here’s two more good reasons you should now operate with a zero trust model when interacting with most online users.

On Reddit, user lotrfan2004 shared a live deepfake video of himself as a completely different person. He claimed this was “extremely easy” to make with Runway and ElevenLabs. (Here’s another recent-ish and believable example.)

Meanwhile on Threads, the generally AI-optimistic Wharton professor Ethan Mollick warned that “You really, really should not trust audio clips anymore” based on a deepfake of his own voice generated from a mere 10-second recording.

OUR HALLUCINATIONS ARE ON YOU

LinkedIn is updating its user agreement to include this banger of a paragraph offloading the responsibility for any inaccurate content created by its own generative AI features to the end user that posts it:

Generative AI Features: By using the Services, you may interact with features we offer that automate content generation for you. The content that is generated might be inaccurate, incomplete, delayed, misleading or not suitable for your purposes. Please review and edit such content before sharing with others. Like all content you share on our Services, you are responsible for ensuring it complies with our Professional Community Policies, including not sharing misleading information.

I made this infographic to explain the minutiae of what’s going on here:

FLAT EARTH PRIMING

For the latest installment of “Alexios wrecks his algorithmic feeds,“ over the past few weeks I have been casually searching for Flat Earth videos on TikTok. My goal was to get a sense of how easily the platform could be turned into a conspiracy conveyor belt. Turns out: Very easily!

On Monday, I took note of the first 100 videos on my For You Page over a single session. Fully 24 were Flat Earth videos. Another 35 videos were about other conspiracy theories. That means that with minor priming, TikTok fed me 59 videos unhinged from reality out of 100. And I’m not alone watching these videos, which had a cumulative 36.3 million views at the time of writing.

Let’s sample some of the material I got to enjoy: