AI-generated romance scams: how deepfakes are changing the game

Real-time deepfake video calls, AI-generated photos and voices. How artificial intelligence is making romance scams harder to detect, and what you can do.

Artificial intelligence has fundamentally changed the romance scam landscape. What once required a scammer to steal real people's photos and laboriously type individual messages can now be automated with tools that generate photorealistic faces of people who never existed, conduct real-time deepfake video calls, clone voices from seconds of audio, and maintain convincing conversations across dozens of victim relationships simultaneously.

This article examines how AI is being used in romance scams, how to detect AI-generated content, and what platforms and law enforcement are doing to respond. The goal is not to create fear but to provide the practical knowledge needed to navigate a world where seeing is no longer believing.

AI-generated profile photos

The first and most widely adopted AI tool in romance scams is the generation of entirely synthetic profile photos — images of people who have never existed.

How it works

Generative AI models — particularly Stable Diffusion, Midjourney, and DALL-E — can produce photorealistic images of human faces based on text prompts. A scammer can request "attractive 35-year-old man in a business suit, professional headshot" and receive a gallery of options within seconds. More advanced techniques use models specifically trained on face generation (descendants of StyleGAN technology) that produce images virtually indistinguishable from real photographs.

Why this matters for romance scams:

• Reverse image search is defeated. Because the person in the photo has never existed, the image will not appear anywhere else online. The traditional advice to "reverse image search their photos" — while still worth doing — will return no results for AI-generated images.
• Unlimited unique identities. A single scammer can generate hundreds of unique, photorealistic profiles. When one profile is reported and removed, a new one can be created instantly.
• Customization to target preferences. Scammers can generate faces that match the specific demographic and aesthetic preferences of their target — age, ethnicity, build, style — with far greater precision than when they had to work with stolen photos.
• Consistency across multiple photos. Advanced tools can generate multiple photos of the same synthetic person in different settings, outfits, and poses, creating a more convincing profile with varied content.

How to detect AI-generated photos

While detection is becoming harder as the technology improves, several indicators can still reveal AI-generated images:

• Background anomalies: AI often generates blurred, distorted, or physically impossible backgrounds. Look for warped architecture, text that does not form real words, objects that merge unnaturally, or patterns that repeat incorrectly.
• Hair and accessory artifacts: Hair strands may merge with the background, earrings may not match, and glasses may have asymmetric frames. Jewelry and accessories are areas where AI frequently produces subtle errors.
• Skin texture: AI-generated faces sometimes have an overly smooth, almost airbrushed quality — particularly noticeable when compared to casual smartphone photos of real people.
• Asymmetry issues: While human faces are naturally slightly asymmetric, AI can produce both unnatural symmetry (too perfect) or inconsistent asymmetry (one eye a different shape from the other in a way that looks wrong rather than natural).
• Teeth: Teeth are historically difficult for generative AI. Look for teeth that blur together, have inconsistent sizes, or display an unusual number (too many or too few).
• Hands and fingers: In full-body or half-body shots, check hands and fingers. AI still frequently produces hands with the wrong number of fingers, unusual joint angles, or fingers that merge together.
• Consistency test: Ask for additional photos. If every photo looks like a professional portrait with a different background but the same perfect quality, it may be AI-generated. Real people have casual, poorly lit, unflattering photos mixed in with good ones.

Real-time deepfake video calls

The most significant AI advancement for romance scammers is the ability to conduct live video calls while wearing a different face. This directly undermines what was previously considered the gold-standard verification method: "Just do a video call."

How it works

Software applications like DeepFaceLive, FaceFusion, and various commercial face-swap apps allow a user to overlay a different face onto their own in real time during a video call. The software processes the webcam feed, detects the user's face, and replaces it with the target face — all with minimal latency. The target face can be:

• A real person's face (using their photos as a source)
• An AI-generated face that matches the profile photos
• A composite face designed to match a specific appearance

The technology has improved dramatically since 2023. Entry-level face swapping requires only a consumer-grade computer with a modern GPU. Higher-quality swaps require more powerful hardware but remain accessible to organized criminal operations.

How to detect deepfake video calls

Despite improvements, deepfake face swaps still have detectable limitations:

• Face boundary artifacts: Look for a visible edge where the swapped face meets the real skin — particularly along the jawline, hairline, and around the ears. This boundary may shimmer, blur, or display inconsistent lighting.
• Head turning: Ask the person to turn their head fully to the left and right. Deepfake overlays often break down at extreme angles, producing distortion, warping, or a visible "seam" where the overlay edge becomes apparent.
• Hands over face: Ask them to wave their hand in front of their face or touch their nose. Objects passing between the camera and the face cause rendering conflicts — the deepfake overlay may flicker, distort, or briefly reveal the real face underneath.
• Lighting changes: Ask them to move to a different room or turn their head toward a window. Sudden lighting changes can expose inconsistencies between the lighting on the swapped face and the lighting on the body and background.
• Unnatural facial expressions: Deepfake overlays can struggle with extreme facial expressions — wide-mouth laughing, exaggerated surprise, or squinting. The overlay may lag behind the real expression or produce unnatural distortion.
• Lip synchronization: Watch carefully for slight delays between audio and lip movement. While modern deepfakes are much better at lip sync, slight discrepancies may still be visible, especially during rapid speech.
• Eye contact: Some deepfake systems struggle with consistent eye tracking. The eyes may appear to look slightly off-camera even when the real person is looking directly at the lens.

AI voice cloning

AI voice cloning technology can replicate a person's voice from as little as three to five seconds of sample audio. Services like ElevenLabs, Resemble AI, and open-source alternatives can produce voice clones that capture accent, tone, pace, and vocal characteristics with remarkable accuracy.

Application in romance scams

• Voice messages: Scammers can send voice messages that sound like the person in their profile photos, reinforcing the illusion that the fake persona is real.
• Phone calls: Combined with deepfake video or used independently, voice cloning allows scammers to make phone calls in a voice that matches the person they are impersonating.
• Voice "verification": When a victim requests proof of identity, a scammer can offer a phone call as verification — using a cloned voice to "prove" they are who they claim to be.
• Emotional manipulation: A voice message saying "I love you" or "I need your help" in a convincing, personalized voice is significantly more emotionally impactful than the same words in a text message.

How to detect voice cloning

• Ask unexpected questions requiring spontaneous speech. Pre-generated voice clips cannot respond to novel questions. If someone can only send voice messages but cannot have a real-time phone conversation, that is suspicious.
• Listen for unnatural pauses or prosody. Cloned voices sometimes have slightly robotic rhythm, unnatural pauses between sentences, or a flatness in emotional tone — particularly when generating longer speech segments.
• Background audio inconsistencies. AI-generated audio may lack natural environmental sounds (room echo, ambient noise) that would be present in a real recording.
• Request real-time conversation about something specific and timely. Ask them to describe what they see outside their window right now, or to react to a news event from today. Real-time, spontaneous speech is very difficult to fake with current voice cloning technology.

AI chatbots and automated conversation

Large language models (LLMs) like ChatGPT, Claude, and their open-source counterparts have given scammers the ability to maintain convincing text conversations with minimal human involvement. This has transformed the economics of romance scams.

Impact on scam operations

• Scale: A single operator who previously managed 5-10 victim relationships can now manage 50 or more, using AI to generate personalized responses and only stepping in for critical moments (financial requests, crisis management).
• Language quality: AI eliminates the grammar errors and awkward phrasing that were once reliable red flags for scams originating from non-English-speaking countries. A scammer who does not speak English natively can now produce flawless, nuanced English text.
• Emotional sophistication: LLMs can generate empathetic, emotionally resonant messages that mimic the patterns of genuine romantic communication. They can adjust tone, express vulnerability, and create the illusion of emotional depth.
• Memory and consistency: AI systems can be configured to remember conversation history and maintain consistent personal details across thousands of messages — something that was difficult for human operators managing many victims simultaneously.
• 24/7 operation: AI does not sleep. Scam operations can maintain round-the-clock communication regardless of time zones, making victims feel they have a dedicated, attentive partner.

How to detect AI-generated messages

• Test with specifics: Ask about very specific local knowledge — a restaurant on a specific street, local slang, recent local events. AI can be good at general knowledge but may struggle with hyperlocal details.
• Look for over-politeness and hedging: LLMs tend to be diplomatic and avoid strong opinions. A person who never has a strong reaction, never disagrees, and always finds the positive spin may be AI-driven.
• Introduce deliberate errors or nonsense. If you type something that does not make sense and receive a smooth, agreeable response that does not acknowledge the confusion, that suggests AI processing rather than human reading.
• Check response latency patterns: AI-generated responses may arrive with remarkably consistent timing — always within 30-60 seconds, regardless of the complexity of the message being responded to.
• Ask for a selfie with a specific action. "Send me a photo of you right now holding up three fingers." A text-based AI cannot produce this; a real person can.

Detection tools and technology

Several tools and services are available to help detect AI-generated content:

• Microsoft Video Authenticator: Analyzes videos and images to provide a confidence score indicating whether the media has been artificially manipulated. Originally developed for election security, it is applicable to deepfake detection in any context.
• Sensity.ai: A platform specifically focused on deepfake detection. It offers API-based analysis that can identify face swaps, AI-generated faces, and manipulated media.
• Social Catfish AI detection: The Social Catfish verification service has integrated AI photo detection into its search tools, flagging images that show characteristics of AI generation.
• Hive Moderation: Offers AI-generated content detection for both images and text, with high accuracy rates for identifying synthetic media.
• FakeCatcher (Intel): Uses biological signals — specifically subtle changes in facial blood flow — to distinguish real video from deepfakes. Blood flow patterns that occur naturally in living faces are not replicated by deepfake generators.
• AI text detectors: Tools like GPTZero, Originality.ai, and Copyleaks can analyze text to estimate the probability that it was generated by an AI language model. While not definitive, they can flag suspicious communication patterns.

What platforms are doing

Major dating and social media platforms are investing in AI-powered defenses against AI-powered scams:

• Meta (Facebook, Instagram): In 2025, Meta began labeling images detected as AI-generated and deployed proactive warnings to users who appear to be in conversation with suspected scam accounts. The company's AI systems analyze conversation patterns for scam indicators and display warnings when red flags are detected.
• Bumble: Bumble's photo verification system requires users to take a real-time selfie that matches a specific pose. The system uses AI to compare the selfie against profile photos. While not deepfake-proof, it adds a meaningful verification layer.
• Tinder: Match Group (Tinder's parent company) has implemented photo verification and is developing AI-based conversation monitoring to detect scam patterns. The company reports blocking millions of suspected scam accounts annually.
• Hinge: Has implemented selfie verification and uses behavioral analysis to identify accounts that exhibit scam-like patterns (rapid messaging to many users, copy-paste messages, geographic inconsistencies).
• WhatsApp: While end-to-end encryption limits content analysis, WhatsApp has implemented account verification features and provides educational pop-ups when users receive messages from unknown numbers.

The scale of the AI-enabled threat

The democratization of AI tools has fundamentally altered the economics of romance scams:

• Lower barrier to entry: Creating a convincing fake persona previously required stealing photos from multiple real people and writing personalized messages in a language the scammer might not speak fluently. Now, anyone with access to freely available AI tools can generate unlimited unique identities and maintain fluent conversations in any language.
• Massive scale: A single scam operator using AI tools can manage dozens of simultaneous victim relationships. Organized operations with multiple operators can maintain hundreds or thousands of active scams concurrently.
• Faster iteration: When a profile is reported and removed, a new one — with a completely different face, name, and backstory — can be generated in minutes.
• Harder detection: The traditional red flags (reverse image search hits, grammar errors, inability to video call) are all neutralized by AI. New detection methods must be developed and adopted continuously.
• Higher success rates: More convincing personas, better language, and the ability to video call all increase the probability that a target will be successfully deceived.

Protection strategies for the AI era

Effective protection against AI-powered romance scams requires a multi-layered approach:

1. Never trust a single verification method. A video call alone is not enough. A photo that passes reverse image search is not enough. Use multiple independent verification methods together.
2. Prioritize spontaneous interactions. Scheduled video calls, pre-recorded voice messages, and carefully composed text all give AI tools time to generate convincing content. Spontaneous, unscheduled interactions are much harder to fake.
3. Verify through offline channels. If possible, verify identity through channels that are difficult to fake: call them at a work phone number listed on a company website, send mail to a physical address, or meet in person in a safe public location.
4. Ask for specific, time-bound proof. "Send me a photo right now holding today's newspaper" or "Take a video saying my name and today's date." These requests require real-time human action that is very difficult to fake even with advanced AI.
5. Trust your instincts about "too perfect." If someone's photos look flawless, their messages are always perfectly crafted, and they never have a bad day or an awkward moment, that level of perfection may itself be a red flag.
6. Use the friend test. Share the profile and conversation with a trusted friend who is not emotionally invested. Outside observers are better at noticing patterns that the target — under the influence of emotional bonding — may miss.
7. Be skeptical of anyone who avoids meeting in person. The ultimate test remains an in-person meeting. A person who has an excuse for every proposed meeting, indefinitely, regardless of how much time has passed or how strong the relationship allegedly is, should be viewed with extreme suspicion.
8. Never send money to someone you have not met in person. This rule is even more important in the AI era. No matter how real someone looks on video, how convincing their voice sounds, or how genuine their messages feel — do not send money to someone you have not physically met.

Sources

• FBI IC3 Public Service Announcement (2023) — "Malicious Actors Manipulating Photos and Videos to Create Explicit Content and Sextortion Schemes"
• Europol Innovation Lab — "Facing Reality: Law Enforcement and the Challenge of Deepfakes" (2024)
• Sensity AI — Annual Deepfake Detection Report 2024
• Microsoft Research — Video Authenticator technology overview and detection methodology
• Meta Transparency Center — AI-generated content labeling policies and scam detection systems
• Match Group Safety Report 2024 — Platform-level fraud detection and prevention measures
• Ajder, H. et al. (2019) "The State of Deepfakes: Landscape, Threats, and Impact" — Deeptrace Labs
• Chesney, R. & Citron, D. (2019) "Deep Fakes: A Looming Challenge for Privacy, Democracy, and National Security" — California Law Review