API Solution

Video Scene Analysis API

AI that watches the video for you. Send a URL and get back what the video shows, scene by scene, with the text written on screen, next to what it says. One JSON, for pipelines and agents that read video at scale.

What is the Video Scene Analysis API?
The Video Scene Analysis API is a REST endpoint, POST /v1/watch, that reads what an online video shows. You send a video URL and receive one JSON with the transcript and a list of scenes, each with its start and end time, a description of the picture, and the text readable on screen.

Why the Video Scene Analysis API?

A transcript is what a video says, and a lot of video says very little: a product demo without a voice-over, a price written on screen, a recipe filmed over music, a clip whose caption says one thing while the footage shows another. A pipeline that only reads transcripts skips those videos without telling you. This endpoint reads the picture as well, and returns both readings in one response.

  • Scenes With a Clock — Every scene comes with its start and end in seconds, a description of the picture, and on_screen_text. You know what was shown, and when.
  • Speech and Picture in One JSON — The transcript arrives next to the scenes, timed on the same clock, so you can tell what was on screen while something was said.
  • Silent Videos Are Answers, Not Errors — A video without speech returns its scenes with an empty transcript and a transcript_error. No special case in your pipeline.
  • On-Screen Text, As Written — Titles, labels, quantities, prices and handles written on the picture come back in on_screen_text, scene by scene.
  • You Choose the Detail, Not a Frame Count — detail takes low, auto or high. The number of frames follows the video: few on a fixed shot, more on a fast-cut clip.
  • Nothing to Operate — No video files to handle, no model to host, no pipeline to keep running. You send a URL and read JSON, from a script, a batch job or an agent.

How It Works

1
Get Your API Key

Sign up and generate an API key in VidNavigator Studio. That is the only credential you need.

2
Send a Video URL

POST the URL to /v1/watch, optionally with detail (low, auto, high), include_transcript and transcript_text. Videos up to 10 minutes are answered in the response.

3
Read the Scenes and the Transcript

The response holds video_info, transcript, scenes and frames_analyzed. Scenes and transcript segments are timed in seconds from the beginning of the video.

4
Go Long, or Go Wide

For videos up to 2 hours, and any length on YouTube, POST to /v1/watch/async: you get a task id at once, then read GET /v1/watch/{task_id} or receive the result on your webhook.

A Real Answer: a Recipe Nobody Reads Aloud

This 78-second recipe video on Dailymotion is filmed over music. Nobody speaks; the ingredients and their quantities are written on screen as they go into the bowl.

This is what a default call returned for it: the data block of the real response, cut short with ... and nothing else changed.

  • The transcript holds what the audio holds: its first segment is the word Música, music.
  • The scenes carry the dish, the steps, and seven ingredient labels read off the screen, quantities included: FRANGO DESFIADO COZIDO 500 g, MAIONESE 300 g, MILHO VERDE 250 g and four more.
  • 30 frames analyzed, 30 scenes, answered in 12 seconds, for about 0.037 credit in total.

The full walk-through is in the guide: What a video shows, next to what it says.

json

Request Parameters

ParameterBehavior
video_urlRequired. The URL of the online video to watch.
detaillow, auto (default) or high: how closely the picture is followed. A frame is taken where the picture changes, and at least one every 10 seconds; detail sets how small a change counts.
include_transcriptDefault true. Set it to false to receive the scenes only.
transcript_textSet it to true to receive the transcript as a single text string instead of timed segments.
include_usageSet it to true to receive what the call was charged.
webhook_urlOn POST /v1/watch/async only. The address called when the job finishes.

What Comes Back

One data block with the metadata of the video, its transcript, and its scenes in chronological order.

FieldContent
video_infoThe metadata of the video: title, description, duration and more.
transcriptWhat the video says, as timed segments, or as one string with transcript_text. Empty when the video has no speech; left out when include_transcript is false.
scenes[].start / endWhen the scene starts and ends, in seconds from the beginning of the video.
scenes[].descriptionWhat is visible in the picture during the scene.
scenes[].on_screen_textThe text readable on screen during the scene. Empty when there is none.
frames_analyzedThe number of frames that were analyzed to produce the scenes. Watching is billed by the frame.
transcript_errorPresent when there is no transcript, for instance on a video without speech. The scenes are returned all the same.
json

The values above are placeholders that show the shape of a response. They are not the output for a real video; the recipe above is.

How Closely Should It Look?

You do not give the endpoint a number of frames. A frame is taken where the picture changes, and at least one every 10 seconds; detail sets how small a change counts. The number of frames therefore depends on the video, and the response reports it in frames_analyzed.

detailUse it for
lowA rough idea of the picture: a talk, an interview, a long video that mostly shows the same thing.
autoThe default. Suits most videos.
highFast cuts, or something brief or small on screen: a slide shown for a moment, a price tag, a logo.

The picture is followed scene by scene, so something that stays on screen for an instant can fall between two frames. When a brief or small detail matters, use high.

Built for Teams That Read Video at Scale

Fact-Checking & Verification

A caption or a post makes a claim about a video; the scenes say what the footage contains, and at which second. Keep description and on_screen_text next to the claim, and a reviewer checks it against the picture without opening the video.

Brand & Market Monitoring

Prices, product names, offers and disclaimers are often written on screen, not said. Watch the videos a keyword search returns and search on_screen_text for the names you track, silent demos included.

Datasets for AI

A scene is already a record: start, end, description, text on screen. Add the transcript segments that fall inside it and you have aligned picture-and-speech rows for retrieval, evaluation or training.

Agents That Read Video

Give an agent the watch_video tool and it answers "what happens in this video?" from the picture as well as the audio, quoting the text on screen and the second it appears.

Quickstart

One request with a video URL. Over HTTP, the answer arrives in the response for videos up to 10 minutes. The Python and JavaScript SDKs, from version 2.4.0, make the same call with watch_video and vn.watch.

The endpoint takes the URL of an online video. Uploaded files are not supported.

API reference
bash

Long Videos, and Many of Them

POST /v1/watch/async takes the same request and returns a task id at once. It accepts videos up to 2 hours, and any length on YouTube.

Read the result on GET /v1/watch/{task_id}, or pass a webhook_url and get called when the job finishes. With the SDKs, submit_watch_video and vn.watch.submit return a job without waiting: submit a list of URLs, then collect the results.

python

For AI Agents

An agent cannot open a video. It can call a tool that reads one and returns text. The scenes are that text: what is shown, the words on screen, and the second they appear.

MCP tool: watch_video

The VidNavigator MCP server exposes the endpoint as the watch_video tool, next to the transcript, search and extraction tools. See the MCP server documentation and the page for AI agents.

Skill: /watch

The /watch skill of github.com/vidnavigator/skills gives a coding agent the same reading: it takes a link and a question, reads what the video says and what it shows, and answers with timestamps.

Supported Platforms

Every platform of the transcript and transcribe endpoints, with the same call:

YouTube
TikTok
Instagram
X
Facebook
Vimeo
Dailymotion
Loom
Snapchat

Pricing

Watching a video is billed by the frames analyzed. With VidNavigator, 1 credit = 1,000 frames.

  • At least 334 frames per hour of video. An hour of video watched never costs less than an hour of transcription.
  • The fetch of the video is charged as for a transcript.
  • Speech-to-text is charged only when the video has no captions.
  • As a guide, at the default level: a fixed shot stays at the minimum, about 0.33 credit per hour of video, and an hour of television programme is typically between 0.5 and 1 credit.
  • You see what you paid. frames_analyzed is in every response, and include_usage returns what the call was charged.

Three measured examples, all at the default level:

VideoLengthFrames analyzedCredits for the frames
Recipe video78 seconds300.03
News programme61 minutes5890.59
Television programme1 h 581,0071.01

With everything else the call is charged for, the recipe came to about 0.037 credit in total. Because the number of frames follows the picture, a long interview on a fixed shot costs little, and a fast-cut clip costs more per minute than a slow one.

Frequently Asked Questions

What is the Video Scene Analysis API?
The Video Scene Analysis API is the POST /v1/watch endpoint of VidNavigator. You send the URL of an online video and receive one JSON with its transcript and a list of scenes. Each scene has a start and an end in seconds, a description of the picture, and the text readable on screen.
What does a scene contain?
Four fields: start and end, in seconds from the beginning of the video; description, what is visible in the picture; and on_screen_text, the text readable on screen, which is empty when there is none. Scenes are returned in chronological order.
How is this different from a transcript API?
A transcript API returns what a video says. This endpoint also returns what it shows. A product demo without a voice-over, a recipe filmed over music, or a price written on screen has little or nothing in its transcript and all of its content in its scenes. Both come back in the same response, timed on the same clock.
What happens with a video that has no speech?
It is not an error. The scenes are returned, the transcript is empty, and transcript_error says why there is no transcript. Set include_transcript to false to receive the scenes only.
Which platforms are supported?
Every platform of the transcript and transcribe endpoints: YouTube, TikTok, Instagram, X (Twitter), Facebook, Vimeo, Dailymotion, Loom and Snapchat. The endpoint takes the URL of an online video; uploaded files are not supported.
How long can a video be?
The synchronous call, POST /v1/watch, takes videos up to 10 minutes. POST /v1/watch/async takes videos up to 2 hours, and any length on YouTube. It returns a task id at once; the result is read on GET /v1/watch/{task_id}, and a webhook_url can be called when the job finishes.
What does the detail parameter change?
How closely the picture is followed: low, auto (the default) or high. At every level a frame is taken where the picture changes, and at least one every 10 seconds; detail sets how small a change counts. The number of frames depends on the video, few on a fixed shot and more on a fast-cut clip, and the response reports it in frames_analyzed.
How is watching a video billed?
By the frames analyzed: 1 credit = 1,000 frames, with at least 334 frames per hour of video, so an hour of video watched never costs less than an hour of transcription. The fetch of the video is charged as for a transcript, and speech-to-text only when the video has no captions. As a guide, at the default level, a fixed shot stays at the minimum, about 0.33 credit per hour of video, and an hour of television programme is typically between 0.5 and 1 credit.
Can AI agents use it?
Yes. The VidNavigator MCP server exposes it as the watch_video tool, and the /watch skill of the vidnavigator/skills repository gives it to a coding agent. The scenes come back as text, so an agent can reason about what a video shows, quote the text on screen and point to the second it appears.

Related Solutions

Related Guides