How to Tell an AI Video Summary Is Fake (Three Checks, Ten Seconds)

Someone sends you a video. You paste the link into ChatGPT and ask what's in it. It answers straight away — clean, organised, confident. You skim it and move on.

Here's how to check whether that answer was real.

Three tells, ten seconds

It's suspiciously even. Real talks ramble. Someone spends nine minutes on one point and thirty seconds on another. If every section in the summary gets roughly equal weight, you're reading an outline, not a talk.

Nothing in it is specific. No numbers. No names. No "she argues that…". Just topics — the subjects that were covered, with nothing that was actually said about them.

It matches the thumbnail. If the summary reads like a longer version of the promise on the thumbnail, that's because it is one.

The single test that settles it: take one claim from the summary and ask whether it could only have come from someone speaking out loud. A number, a name, a moment where the argument turns. If nothing in the summary passes that test, nothing in it came from the video.

Why this happens

It isn't your prompt, and no rewording fixes it. A chat model handed a video URL doesn't watch anything — it reads the title and the description, which is marketing copy the creator wrote to get you to click, and summarises that instead.

What makes this worse than an ordinary wrong answer is that it doesn't look wrong. A hallucination usually reads a bit off. This reads perfect, because the blurb was written by a human to be readable. You get a confident, well-structured summary of an advertisement, and no signal at all that you should doubt it.

What to do instead

Give it the words. Get the transcript of the video first, then hand the model the transcript rather than the link — at that point it's doing the thing it's genuinely good at, which is reading a lot of text and telling you what's in it.

Or skip the middle step: paste the link into a transcript tool and read the summary it produces from the actual spoken words.

I wrote up the longer version of this — including the prompt that stops a model padding a transcript summary with general knowledge, and what to do about the videos ChatGPT has no description to fall back on at all — over here: why ChatGPT still can't read YouTube links.