u/Western-Chemistry-40

[ Removed by Reddit on account of violating the content policy. ]

I have been working on this problem for a while and want to think out loud with people who actually understand it, because I am not convinced the mainstream approach is the right one.

Most deepfake defense today is detection: analyze the media, score how likely it is fake, flag it. My worry is that this is a losing arms race. Every time detectors get better, generators get better, and the defender is always reacting to the last model. By the time you have analyzed a call, the wire transfer already went out.

So the approach we are taking flips it. Instead of asking "is this media fake," we ask "is this the person they have always been." The bet is that identity and relationship history are a more durable signal than the pixels or the audio.

Roughly how it works:

• A device-level identity key travels with every communication and builds verified history over time. The more two parties interact and confirm each other, the stronger the trust.
• Trust is tiered. Someone goes from an unknown device fingerprint up to a fully verified coworker through actions like confirming email and repeated real interaction. It is earned, not assumed.
• The user never sees a dashboard or a confidence score. They see one signal during a call: green, yellow, or red. Trust, caution, stop.

The thinking is that an attacker can clone a face and a voice, but cloning a long history of verified relationship on a specific device is a much harder problem.

Where I want this sub to push back:

• Is relationship history actually more durable, or am I just moving the attack surface to device and account takeover?
• A green light could create dangerous overconfidence. If people stop scrutinizing anything green, does the system make them more vulnerable on the day it is wrong?
• New legitimate contacts always start untrusted. Does the friction kill it in practice, especially for roles that talk to strangers all day (sales, recruiting, support)?
• Is "restore trust" just "detection" with extra steps, or is it meaningfully different?

Quick reframe that I think gets lost in every "AI deepfake" headline: for most organizations the realistic threat is not mass disinformation or some viral fake video. It's one cloned voice calling your finance person and saying the CEO needs a wire moved today. 

That attack doesn't need a good deepfake. It needs a few seconds of audio (which your execs hand out for free on every podcast and earnings call) and an org with no verification step. The tech got cheap, but the thing it's exploiting is the same thing BEC has always exploited: authority, urgency, and no callback procedure. 

Here's the part people get wrong. Everyone wants to buy a detector. Detection on raw pixels or audio is an arms race the defender is mostly losing, because the attacker gets to iterate against your detector until it passes. 

Where serious people are putting their attention now is provenance, not detection. Signing content at capture, content credentials, that whole direction. The honest problem there is adoption, it only helps if enough of the ecosystem signs and enough people check, and we are nowhere near that yet. 

So if detection is shaky and provenance isn't widespread, what actually moves the needle today is boring process stuff: 

• out-of-band verification for anything involving money or credentials. callback on a known number, not the number that just called you. a code word for high-value requests. it sounds dumb until it saves you six figures. 
• assume your execs' voices are already clonable, because they are. train people that "it sounded exactly like him" is not evidence of anything anymore. 
• the threat-model question first: who would actually target you, for what, and is a real-time video swap on a Teams call realistic for your org, or is the realistic version just a voicemail and a fake invoice? for most of you it's the cheap version.

 

The other thing nobody likes talking about is the liar's dividend. The flip side of fakes being plausible is that real footage now gets dismissed as fake. That's a whole separate headache for anyone who relies on recordings as evidence, and I don't think we've reckoned with it. 

Curious where everyone else has landed on this: 

• has anyone actually deployed deepfake detection in production and found it worth it, or did it go stale on you? 
• what's your org's actual verification step for wire/credential requests, if any? "we don't have one" is a valid and depressingly common answer. 
• are you seeing more of the targeted fraud flavor or the disinformation flavor in your world?

 

 

Been chewing on this and I want to know where this sub lands, because the more I read the less convinced I am that detection alone ever "wins."

The basic problem: most detectors are trained to spot artifacts that current generators leave behind. Weird blinking, edges that do not quite sit right, audio that is a little too clean, lighting that does not match. The catch is that every one of those tells is just a training target for the next generation model. You publish a detector, the generator learns to beat it, and you are back to square one. It is less a solved problem and more a treadmill.

A few things that stand out to me:

• Detection accuracy in a lab on a known dataset is very different from accuracy in the wild on a compressed, re-uploaded, screen-recorded clip. The numbers in the press releases rarely survive contact with reality.
• Real-time matters more than people admit. A detector that needs to analyze a full file is useless against a live video call asking you to approve a payment right now.
• The base rate problem is brutal. If real communications massively outnumber fakes, even a very accurate detector throws enough false positives to make people stop trusting the flag.
• Provenance approaches (signing and watermarking content at creation, like C2PA) are promising but only help when the source cooperates. Attackers will not watermark their fakes.

So a few questions for the room:

• Has anyone here actually seen a detector hold up against the newest open models, or does everything degrade fast once the generator is even slightly novel?
• Do you think the future is detection, provenance, identity verification, or some stack of all three?
• For the non-technical people in your life, what advice actually works? "Look for blinking" feels obsolete already.

Curious whether anyone here is optimistic about pure detection, or if the consensus is that it is one layer and never the whole answer.

I have been working on this problem for a while and want to think out loud with people who actually understand it, because I am not convinced the mainstream approach is the right one.

Most deepfake defense today is detection: analyze the media, score how likely it is fake, flag it. My worry is that this is a losing arms race. Every time detectors get better, generators get better, and the defender is always reacting to the last model. By the time you have analyzed a call, the wire transfer already went out.

So the approach we are taking flips it. Instead of asking "is this media fake," we ask "is this the person they have always been." The bet is that identity and relationship history are a more durable signal than the pixels or the audio.

Roughly how it works:

• A device-level identity key travels with every communication and builds verified history over time. The more two parties interact and confirm each other, the stronger the trust.
• Trust is tiered. Someone goes from an unknown device fingerprint up to a fully verified coworker through actions like confirming email and repeated real interaction. It is earned, not assumed.
• The user never sees a dashboard or a confidence score. They see one signal during a call: green, yellow, or red. Trust, caution, stop.

The thinking is that an attacker can clone a face and a voice, but cloning a long history of verified relationship on a specific device is a much harder problem.

Where I want this sub to push back:

• Is relationship history actually more durable, or am I just moving the attack surface to device and account takeover?
• A green light could create dangerous overconfidence. If people stop scrutinizing anything green, does the system make them more vulnerable on the day it is wrong?
• New legitimate contacts always start untrusted. Does the friction kill it in practice, especially for roles that talk to strangers all day (sales, recruiting, support)?
• Is "restore trust" just "detection" with extra steps, or is it meaningfully different?

Spent the last while going deep on deepfake driven social engineering and the part that stuck with me is how the threat has quietly shifted. It is not just fake celebrity videos anymore. It is a voice or a face on a work call that looks and sounds exactly like someone you know, asking you to move money or hand over access. The detection-after-the-fact model feels broken, because by the time you are analyzing a clip, the damage is usually done.

What I keep landing on is that the real problem is not "spot the fake," it is "rebuild trust in the channel." A few questions I would love this sub's take on:

• Do you think pixel-level deepfake detection can keep up, or is it a losing arms race against better generators?
• Is identity and relationship history (proving the person is who they have always been) a more durable signal than analyzing the media itself?
• For anyone working in this space, are continuous multimodal signals (voice, video, device, behavior together) actually better than any single detector?

For transparency, I have been looking at this through the lens of a platform called Netarx, which leans on the trust-the-relationship approach rather than pure detection: a device-level identity key that builds verified history over time and boils everything down to a simple green / yellow / red signal during a communication. Not trying to pitch it, more interested in whether people here think that framing (restore trust vs detect fakes) is the right direction or a dead end.

What is the community's read?