Understanding Deepfakes and the Authenticity Challenge

In 30 seconds, AI showed me patrolling Midtown, flexing with a private jet, and watching sunset at the Pyramids—yet I never left my chair. As a company that builds this technology, we watch clips like these every single day. Some are brilliant; some are unsettling. Over time, we’ve learned to spot the giveaways—tiny lighting glitches, off‑beat lip movements, location jumps you only notice on the tenth viewing. To share that hard‑earned pattern‑recognition, we’ve compiled the State of Deepfakes Report: What’s inside? A field guide to the four generations of deep‑fake video, from simple face swaps to the near‑flawless, long‑form scenes now emerging. The tell‑tale signs for each generation—what breaks first when reality is synthetic. Real‑world misuse cases we’ve already encountered, and the safeguards that work (or fail). Why publish it? Because the people best positioned to expose the risks are the ones building the tools. Because watermarking and provenance standards aren’t universal—yet. Until they are, collective awareness is our strongest defense. If AI can make me do all that on screen, it can fabricate a confession, a crisis update, or a world‑shaking headline just as easily. Knowing what’s technically possible—and what subtle errors still slip through—helps all of us judge what we see with clearer eyes. 🔗 The full report is linked in the first comment. Read it, share it, and let’s keep the conversation grounded in facts, not hype. #Deepfakes #ResponsibleAI #MediaIntegrity #Transparency #PublicSafety

#Misinformation and #deepfakes are a HUGE concern when using AI models. Why? AI models are prone to hallucination (read: make things up /or be convincingly wrong.) AI is also being used by bad actors to create realistic misinformation with malicious intent. From rappers to political candidates, authentic-sounding deepfakes persuade us to believe or act in ways inconsistent with the way we would with accurate information. Case in point - the 2024 US Presidential election. No stranger to controversy, the next one stands to test Americans’ collective Internet patience. What should we watch for? - Disinformation: the deliberate creation and/or sharing of false information in order to mislead; - Deepfakes: a type of disinformation that uses AI to create realistic but fake audio or video content; and - Misinformation: the act of sharing information without realizing it’s wrong. How do you know if the info you see online is real? The answer lies in due diligence. Take extra steps like these to help ensure that you’re not spreading misinformation, or falling prey to deepfakes and disinformation: - To spot a deepfake, look for isolated blurry spots in the video, double edges to the face, changes in video quality during the video, unnatural blinking or no blinking, and changes in the background or lighting. - Check the source of the information! If you’re using an AI, ask it to list all URL sources (or direct and general sources for models not currently connected to the Internet, like #ChatGPT, #GPT4 and #Claude2.) - Look for other sources that confirm or refute the information. - Check if the information is being reported by reputable news organizations. - Be wary of sensational headlines. - Check if the information is being shared out of context. - Be skeptical of images and videos that seem too good to be true. (It’s time to turn the BS meter way, way up!) What’s your comfort level in spotting disinformation and deepfakes? Do you use any detection tools? Reply in comments - #ai #llm #genai #aiethics #aibias #aiart #promptengineer #generativeai #conversationalai #deepfakes #misinformation #disinformation

I can't kick the feeling that the primary issue...the issue that aims to *invariably* jeopardize the future is dis/misinformation / weaponized media. Whatever societal or global problems we need to tackle, we need to start from common-ground, right? Our attention, taste, preferences, judgment, etc are increasingly engineered to great effect. Not by just one person or one group or one company. Many benign actors but more and more bad ones are pushing the trend for different reasons. Deepfake OSS tools are staggeringly powerful and accessible. If you have a few hours to spare and are sufficiently motivated, you can compose shockingly believable media of all kinds (audio, video, obv prose, etc) that are extremely hard to pick out and growing more believable every day. A solution will have to come from serious collaboration across people, companies, gov'ts, technologies, and disciplines. E.g. every device capable of capturing or transmitting anything needs to be able to cryptographically verify content at various interaction points. Just as HDMI transmits hi def audio and video and much more, well, surely a new authenticity/verification protocol is needed alongside other signals and data. Lat/lon on photos, totally original photorealistic augmented realities in pure CGI and/or based on actual footage all can be faked and manipulated, no problem. We need something like immutable proof signatures. News or political debates need something like a tamper-proof watermark. And we users need easy-to-understand design idioms that empower us to know what’s real at a glance.

We can't be surprised by this, and content moderation will only go so far when trying to mitigate this kind of disinformation. This will complicate complying with (and auditing for) the Digital Services Act. "Days before a pivotal national election in Slovakia last month, a seemingly damning audio clip began circulating widely on social media. A voice that sounded like the country’s Progressive party leader, Michal Šimečka, described a scheme to rig the vote, in part by bribing members of the country’s marginalized Roma population." "Rapid advances in artificial intelligence have made it easy to generate believable audio, allowing anyone from foreign actors to music fans to copy somebody’s voice — leading to a flood of faked content on the web, sewing discord, confusion and anger." "On Thursday, a bipartisan group of senators announced a draft bill, called the No Fakes Act, that would penalize people for producing or distributing an AI-generated replica of someone in an audiovisual or voice recording without their consent." "Social media companies also find it difficult to moderate AI-generated audio because human fact-checkers often have trouble spotting fakes. Meanwhile, few software companies have guardrails to prevent illicit use." "In countries where social media platforms may essentially stand in for the internet, there isn’t a robust network of fact-checkers operating to ensure people know a viral sound clip is a fake, making these foreign language deepfakes particularly harmful." #disinformation #deepfake #aiethics Ryan Carrier, FHCA, Manon van Rietschoten, Dr. Benjamin Lange, Maurizio Donvito, Mark Cankett https://lnkd.in/daRx25sf

Beyond Deepfakes: The Hidden Dangers of AI-Enhanced Evidence in Court As a judge deeply immersed in the intersection of law and technology, I am acutely aware of the profound challenges and opportunities that AI and digital evidence present to our legal system. For some time now, I have been vocal about the potential dangers of deepfakes—AI-generated videos and audio clips designed to deceive by fabricating events that never occurred. However, another challenge is also emerging in our courtrooms: the use of professionally enhanced evidence. While these enhancements are often presented transparently as attempts to clarify, they could still mislead and distort the truth. The Threat of Deepfakes Deepfakes represent a significant threat in legal contexts because their primary purpose is to deceive. These AI-generated videos can convincingly depict individuals doing or saying things they never did, posing a severe risk to the integrity of evidence. My concern has always been that without careful scrutiny, the justice system may struggle to distinguish fact from fiction. The intentional creation of false realities can fundamentally undermine our legal processes, making it difficult for courts to ensure that justice is based on truthful representations. The Challenge of Professionally Enhanced Evidence In contrast, professionally enhanced evidence is usually presented with the intention of improving clarity and aiding understanding. However, this transparency does not eliminate the risk of distortion. The recent ruling by Judge Leroy McCullough in Washington state, where AI-enhanced video was ruled inadmissible in a murder trial, underscores this issue. Judge McCullough highlighted that the AI technology used "opaque methods to represent what the AI model 'thinks' should be shown," that could have lead to a potential misrepresentation. The judge also noted that considering such evidence could have “lead to a time-consuming trial within a trial about the non-peer-reviewable-process used by the AI model." Although the enhancements aimed to clarify, they risked introducing inaccuracies, demonstrating that even well-meaning enhancements can deceive. A Historical Perspective: Zooming in on Photos The legal system's struggle with enhanced evidence is not new. Years ago, we faced similar issues when people began zooming in on photos to extract details. At first, this seemed like a straightforward way to clarify evidence. However, it soon became clear that zooming in could also introduce distortions ... Continue reading article here: https://lnkd.in/gQbEPiA8

Generative and Deep Fake Fraud on the Rise, my canary in the coal mine is getting sick! Every time I am hanging out with our amazing merchant members I always ask the question is anyone experiencing fraud attacks using deep fakes or generative AI. While fraudsters have been using FraudGPT and other unlocked AI Engines to do bad things like write better phishing emails, assist with writing code for bots and other fairly obvious things, until this month I had not heard of real and consistent scams leveraging these new tools. This simple question has been my canary in the coal mine. This past few weeks I have now had two merchants give me very real examples where the criminals are leveraging these new tools in innovative and more sophisticated ways. In one example the fraudster used audio samples to create a fake audio of the consumer to bypass voice biometric authentication. In another the fraudster used the deep fake tech to create a video of themselves with some required docs to pass screenings. What does this mean? In a nutshell it means as I have been saying for 25+ years: - there is no silver bullet to solve fraud - every awesome fraud tool works really well until it is adopted in the mainstream and then fraudsters crack it and it’s effectiveness diminishes, it’s still valuable, just not as valuable To me this means the biometric of voice and video are on the edge of starting to be compromised, the fraud detection tools need to get more sophisticated, the fraudsters will improve their attacks too. Be alert, if you use voice or video as part of your fraud screening, train your team to be alert for the fakes, I believe it’s still pretty obvious through manual review but it’s here. I played Christmas music on the drive home last night, because as I pondered what this means, I realized Black Friday/Cyber Monday is right around the corner and we are going to have to collaborate and stay connected this season as this is a very real new threat merchants need to be prepared for and it’s going to appear in ways we never thought of.

New article 🚨. Generative AI and deepfakes reinforce deep-seated human rights and societal challenges around trust, evidence and efficacy at the same time as audiovisual media and technologies play a central role in contemporary communication and human right investigatory and advocacy strategies. In Journal of Human Rights Practice I explore critical proactive steps to 'fortify the truth', drawing on WITNESS ongoing work, and looking at tactical, strategic and technical steps across filming, sharing, storytelling, archiving and advocacy. 👊 Audiovisual digital media and tools are critical elements in contemporary human rights documentation and advocacy. 🤦♂️ Generative AI, deepfakes and synthetic media compound questions of what to trust in an existing situation of government suppression, difficulty proving witness accounts and broader societal challenges to trust. 🌍 There is a need to ‘fortify the truth’ by fostering resilient witnessing practices that can ensure trustworthy videos and strengthen narratives of vulnerable communities. ⚒ This requires actions at tactical, strategic, tools, technology, and policy levels 🔰 We can learn from WITNESS’s work on proactive preparation for emerging technologies and technical infrastructures including our 'Prepare, Don't Panic' work on deepfakes, and our proactive work on democratizing access to critical skillsets. 📹 Practical steps occur across a trajectory of using images and video in human rights advocacy and activism including filming, storytelling, watching, analysing, sharing, advocacy, and preservation. 📷 Guidance on filming must evolve to address deepfakes and opportunities and challenges in ‘authenticity infrastructure’. 😓 Narrative video advocacy and formal legal and policy processes must adapt to new technologies including text-to-image and text-to-video, new disinformation threats such as ‘floods of falsehood’ and new presentation opportunities. 📱 The evolution of watching, scrutinizing, and sharing videos accountability amid increasing volume and normalized image manipulation includes positive dimensions of the ‘media forensic turn’, including collaborative ‘open-source intelligence’ verification, and negative aspects involving excessive scrutiny. 👉 Preserving audiovisual media is critical, and emerging socio-technical infrastructure should be shaped for community control. ⏰ Underlying principles for ‘fortifying the truth’ include taking a proactive approach, centring voices and needs of people facing human rights abuses, and working with and challenging technologists and technology companies. https://lnkd.in/eBCanrXc #deepfakes #AI #generativeAI #humanrights

I think l am loosing my capacity for trust… Generative artificial intelligence is now capable of creating fake pictures, clones of our voices, and even videos depicting and distorting world events. The result: From our personal circles to the political circuses, everyone must now question whether what they see and hear is true. We’ve long been warned about the potential of social media to distort our view of the world, and now there is the potential for more false and misleading information to spread on social media than ever before. Just as importantly, exposure to AI-generated fakes can make us question the authenticity of everything we see. Real images and real recordings can be dismissed as fake. “When you show people deepfakes and generative AI, a lot of times they come out of the experiment saying, ‘I just don’t trust anything anymore,’” says David Rand, a professor at MIT Sloan who studies the creation, spread and impact of misinformation. This problem, which has grown more acute in the age of generative AI, is known as the “liar’s dividend,” says Renee DiResta, a researcher at the Stanford Internet Observatory. The combination of easily-generated fake content and the suspicion that anything might be fake allows people to choose what they want to believe, adds DiResta, leading to what she calls “bespoke realities.” Examples of misleading content created by generative AI are not hard to come by, especially on social media. One widely circulated and fake image of Israelis lining the streets in support of their country has many of the hallmarks of being AI-generated—including telltale oddities that are apparent if you look closely, such as distorted bodies and limbs. For the same reasons, a widely shared image that purports to show fans at a soccer match in Spain displaying a Palestinian flag doesn’t stand up to scrutiny. The signs that an image is AI-generated are easy to miss for a user simply scrolling past, who has an instant to decide whether to like or boost a post on social media. And as generative AI continues to improve, it’s likely that such signs will be harder to spot in the future. Is Anything Still True? On the Internet, No One Knows Anymore https://lnkd.in/dACHjeUM

It's time to think about "Deepfakes: Opportunities, Risks, and Regulation" - my latest article just got published in Predict! >> I share an overview of the current deepfake landscape, from opportunities to risks, as well as risk management options and regulation adjustments. >> I offer a framework to think clearly about deepfake management and regulation, based on identifying segments of intentions and results: #MalevolentAgents with bad intentions: hackers, scammers, cybercriminals, who ask for ransom, or look to destroy your reputation, conduct propaganda and misinformation, and even terrorize populations… #RuthlessActors: they believe it is fun to play with the technology without thinking through the consequences on misinformation, copyright infringement, or international diplomacy. #TheGreyZone: they don’t necessarily mean bad, but if it’s good for business, they close our eyes on the bad aspects. They might not protect their data enough, or they might reproduce institutionalized bias. #UnintendedConsequences: You mean good, but bad happens, for example artists accept to deepfake their image and performance against remuneration — in that sense they harness their identity, but as a consequence their image loses its rarity. #InsufficientPerspective, which comes with unforeseen consequences. We have a hard time figuring out what could be the biggest risks of technology on child development, mental health, cognitive, social, and emotional development, on behaviors… because we don’t have enough distance, experience, and clinical data. (#ProteusEffect, Her phenomenon) >> I scanned for signals, from the EU AI Act and the No AI FRAUD Act, to McAfee #ProjectMokingbird, Google fighting deepfakes in election campaigns, Intel combatting bias, and more generally the rising cat-and-mouse game between counter-measures and counter-countermeasures! Read more on #Medium! Silicon Humanism Grey Swan Guild In Conversation Informing Choices #deepfake #generativeAI #ethics #AIethics #AIregulation #AIAct #NoAIFRAUD #foresight #futures #identityeconomy #siliconhumanists