Solution

AI Video Analysis & Q&A with Timestamped Evidence

Ask a video for a summary, the people and places in it, or the answer to your question, in plain words. Every value comes back with the moments of the video it was taken from. API and no-code dashboard.

What is the Video Analysis API?
The Video Analysis API is VidNavigator's extraction endpoint, POST /v1/extract/video, used to summarize a video and answer questions about it. You say what you want in plain words and get JSON: a summary, the people and places mentioned, the answer to your question, each with the moments of the video it comes from.

Highlights

  • Summaries, people, places and answers to your questions, asked for in plain words.
  • Every value comes with its evidence: the seconds of the video it was said or shown.
  • With include_scenes, what the video shows is read too: text on screen, products, people.
  • Online videos by URL with /v1/extract/video, your uploaded files with /v1/extract/file.
Ask in plain words

Write what you want: a summary, the people mentioned, the answer to a question. Each becomes a field of the JSON, and the schema written for it comes back to reuse on every video.

Answers with their evidence

Each value comes with the moments it was taken from, start and end in seconds, and whether it was said, shown on screen or read in the post's own text.

Speech and picture

With include_scenes, what the video shows is read with what it says: a price on a label or a name on a lower third is found even when nobody says it.

What data is returned

One JSON per video or file, with the fields you asked for and where each one comes from:

  • data: the fields you asked for, such as a summary, people, places or the answer to your question
  • evidence: for each field, the moments it comes from, with start and end in seconds
  • schema: the JSON Schema written from what_to_extract, when you sent none
  • video_info: title, channel, duration, views, publish date and thumbnail of the video
  • file_info, on /v1/extract/file: name, type and duration of your file
  • frames_analyzed, with include_scenes: the number of frames read
  • scenes_error or transcript_error: when the picture or the speech could not be read, the answer uses the other
  • usage, with include_usage: what the call was charged

API Endpoints

Two endpoints for different sources:

/v1/extract/video

Any online video, by URL. The captions are used when they exist, and the audio is transcribed when they do not.

  • •what_to_extract in plain words, a JSON Schema, or both
  • •include_scenes to read what the video shows as well
  • •Up to 10 minutes per call; /v1/extract/video/async with a webhook for longer videos
View /extract/video docs →

/v1/extract/file

Your uploaded files, by file ID: meeting recordings, podcasts, courses and other private content.

  • •Same request and response as /extract/video, with a file_id
  • •Works on any uploaded audio or video file once it is transcribed
  • •Pairs with /search/file: find the right files, then ask them your questions
View /extract/file docs →

Code examples

Send a video URL and say what you want. You get back the fields, the schema written for them, and the evidence of each value.

Add "include_scenes": true to read what the video shows as well: the response then has frames_analyzed, and evidence can point to the picture.

bash

Response example

The field names come from the schema written for your request, which is returned with the answer.

{
  "status": "success",
  "data": {
    "summary": "The host reviews the Aero X2 earbuds after two weeks of daily use. Battery life and call quality are praised, the fit is the main complaint. The verdict: a good buy for commuters, less so for runners.",
    "people": ["Maya Chen, the host", "Leo Park, the audio engineer she interviews"],
    "places": ["Seoul, where the earbuds are designed"],
    "quoted_price": "$299, or $249 with the launch discount"
  },
  "evidence": {
    "summary": [{ "start": 0.0, "end": 38.5, "in": "speech" }, { "in": "post" }],
    "people": [{ "start": 4.0, "end": 11.2, "in": "speech" }, { "start": 212.3, "end": 220.0, "in": "speech" }],
    "places": [{ "start": 230.1, "end": 236.4, "in": "speech" }],
    "quoted_price": [{ "start": 401.2, "end": 409.8, "in": "speech" }]
  },
  "schema": {
    "type": "object",
    "properties": {
      "summary": { "type": "string", "description": "A three-sentence summary of the video" },
      "people": { "type": "array", "description": "People mentioned", "items": { "type": "string", "description": "A person and who they are" } },
      "places": { "type": "array", "description": "Places mentioned", "items": { "type": "string", "description": "A place and why it is mentioned" } },
      "quoted_price": { "type": "string", "description": "The price quoted for the product" }
    }
  },
  "video_info": {
    "title": "Aero X2 review: two weeks later",
    "channel": "Gadget Desk",
    "duration": 655,
    "views": 48200
  }
}

No-Code Dashboard

Use the VidNavigator Studio at /studio/analyze to analyze videos without writing code:

1

Paste any video URL

Drop a YouTube, TikTok, Instagram, or other video link and get instant analysis.

2

View structured insights

See summaries, people, places, and topics in a clean interface.

3

Ask questions

Type any question and get AI-powered answers with timestamped segments.

4

Export data

Download the full transcript or copy the JSON analysis for your own projects.

How it works

  1. 1.You send a video URL or a file ID, and what you want in plain words or as a schema
  2. 2.The captions are read, or the audio is transcribed when there are none
  3. 3.With include_scenes, what the video shows is read as well
  4. 4.You get JSON: your fields, and the moments each one comes from
OutputWhat it contains
SummaryAsk for "a three-sentence summary": evidence points to the passages it sums up
People and placesAsk for "the people and places mentioned, with who or what they are": a list for each
Answer to a questionAsk for "the answer to: what price is quoted?": the answer, with the seconds it was said or shown
Your own fieldsAny JSON Schema: strings, numbers, booleans, enums, arrays and nested objects
Evidencestart and end in seconds, and in: speech, picture (with include_scenes) or post

Try it

FAQ

Which video links are supported?
YouTube, TikTok, Instagram, Facebook, X, Vimeo, Dailymotion, Loom, Rumble and Snapchat, by URL with POST /v1/extract/video. Your own audio and video files go through POST /v1/extract/file, with the file_id of the upload.
How do I get a summary, or ask a question about a video?
Say it in plain words in what_to_extract, for example "a three-sentence summary, the people and places mentioned, and the answer to: what price is quoted?". A schema is written from it and returned as schema beside data: pass it back as schema to get the same fields from every video. You can also send your own JSON Schema from the start.
How do I know where an answer comes from?
Every answer carries evidence beside data: for each field that has a value, the moments of the video it was taken from, with start and end in seconds, and whether it was said (speech), shown or written on screen (picture, with include_scenes) or read in the post's own text (post).
Do I need a transcript?
No. The platform's captions are used when they exist. When they do not, the audio is transcribed with speech-to-text, which is the default (transcribe is true). Set transcribe to false to use existing captions only.
Can the answer use what the video shows, not only what it says?
Yes, with include_scenes set to true. The scenes of the video and the text on screen are read together with the transcript, so a price on a label or a name on a lower third counts as much as what is said. The response then carries frames_analyzed, and the frames are billed at the price of /v1/watch (see pricing). To get the scenes themselves, use POST /v1/watch.
What's the difference between /extract/video and /extract/file?
/v1/extract/video takes the URL of an online video. /v1/extract/file takes the file_id of an audio or video file you uploaded, once it is transcribed. Both take what_to_extract, schema and include_scenes, and return the same response. For a file, include_scenes uses the scenes kept with the file: ask for them at upload, or later with POST /v1/file/{file_id}/scenes.
What about long videos?
POST /v1/extract/video takes videos up to 10 minutes and answers in the same call. Longer videos go to POST /v1/extract/video/async, which returns a task id at once and can call your webhook when the job finishes. The Python and JavaScript SDKs run extractions as background jobs for you.
Can I download the transcript?
Yes. In Studio, Analyze Video shows the transcript and lets you download it. Through the API, POST /v1/transcript returns it (POST /v1/transcribe for speech-to-text), and POST /v1/watch returns it next to the scenes.
Can I use the output for RAG?
Yes. The answer is JSON with the fields you asked for, and evidence gives the seconds each value comes from, so you can store both and link every value to the moment of the video it is taken from.

Related