Gemini Now Picks Which Parts of a Video to Watch

Introduction

On September 1, 2026, Google shipped agentic video understanding for Gemini — a change in how the model reads video, not a new model. Instead of sampling a file at a fixed frame rate and paying for every frame, Gemini runs an agentic loop that decides what to watch, at what speed, and through which modality: frames, audio or transcript.

Google reports up to 88% fewer tokens, up to 66% lower cost and up to 7% higher accuracy on standard video analysis benchmarks, with the gains largest on long-form content — its examples run from ten-minute how-to guides to 90-minute lectures and multi-hour recordings.

What an hour of video actually costs

The savings only make sense against the baseline, which the Gemini API documentation states plainly. In the default static mode, video is sampled at 1 frame per second and costs roughly:

• ~100 tokens per second of video at low media resolution
• ~300 tokens per second at high media resolution
• +32 tokens per second of audio, processed at 1 Kbps

So a one-hour recording with its soundtrack, at low resolution, is about 475,000 input tokens — 360,000 for frames plus 115,200 for audio. On Gemini 3.7 or 3.8 Flash at the introductory rate of $0.75 per million input tokens (it doubles to $1.50 on January 1, 2027), that single hour costs about $0.36 before the model has produced a word. Video and audio tokens are billed at the same rate as text.

Apply Google's 88% ceiling and the same hour drops to roughly 57,000 tokens, around $0.04. That is arithmetic on Google's own published numbers, not a measured result — but it is the right order of magnitude to plan against, and it is the difference between "we sampled a few videos" and "we indexed the archive."

The token math also explains a limit that looks arbitrary in the docs: 3 hours is the maximum video length at low resolution on a 1M-token model, and 1 hour at high resolution. Those are not policy caps, they are the context window divided by 100 and 300 tokens per second. Whether agentic mode raises the ceiling, Google does not say — it only says the mode loads segments on demand.

What actually changed

Fixed-rate ingestion is wasteful in an obvious way: a one-hour lecture where the answer lives in a 20-second stretch still costs the full hour. Developers have long worked around it by hand — trimming clips, lowering fps, pre-filtering with a transcript. Google's framing is that the model now does that work itself, "invoking an internal tool to load the relevant part of the video file."

That makes it a tool-use loop rather than a preprocessing trick, the same pattern as any other agentic workflow: plan, call a tool, look at the result, decide whether more is needed. Four capabilities follow:

• Sub-second moment retrieval — locating a precise timestamp, accurate enough for automated video editing.
• Needle-in-a-haystack search across multi-hour recordings.
• Anomaly detection by resampling a suspicious stretch at a higher frame rate than the 1 FPS baseline.
• Counting repeated actions or objects over time, which fixed sampling routinely gets wrong when events fall between frames.

The third is the tell. Dynamic resampling means the model can spend more compute on a short segment than static ingestion ever would, while spending far less overall — the budget moves rather than shrinking uniformly.

Availability

Available now through the Gemini API in Google AI Studio and the Gemini Enterprise Agent Platform. Google's announcement named Gemini 3.7 Flash, 3.6 Flash and 3.5 Flash-Lite; the API documentation also lists Gemini 3.8 Flash, which Google released on September 2, 2026, one day later — the third Flash release in three months. Uploaded files and YouTube URLs both work, the Flash and Flash-Lite models are rolling the capability out to Gemini app users, and YouTube's "Ask YouTube" feature is to follow.

Enabling it is one field: set processing to "agentic". There is no surcharge — you pay standard token pricing for whatever the model ends up loading, which is the point. The bill becomes a function of how much video the model decides it needs, not how much you handed it.

The parts Google left vague

• One benchmark, and it stops at an hour. The post charts Gemini 3.7 Flash with and without agentic understanding on LongVideoBench — 6,678 questions over 3,763 videos, none longer than an hour — and claims the best "accuracy-to-cost pareto frontier" overall. But the release is sold on multi-hour recordings, and the only named benchmark does not reach them. The three headline numbers are attributed to "standard video analysis benchmarks," unnamed, all three prefixed "up to."
• No latency figures. An agentic loop adds round trips. Whether a 20-second retrieval over a 90-minute file returns faster or slower than one fixed-rate pass is unanswered, and for interactive products that is the question that decides adoption.
• No cost variance. Spend now depends on the model's own decisions, so the same query over the same file need not cost the same twice. Budgeting a fixed-rate pipeline is arithmetic; budgeting this is forecasting.
• No accuracy floor. A model that chooses what to skip can skip the wrong thing, and "up to 7% better" on average says nothing about the tail.

Conclusion

This is an unglamorous efficiency release, and the shape is familiar: the expensive part of multimodal work is not the model's intelligence but the volume of tokens the input turns into. At 100 tokens per second, video is the most token-hungry input the API takes, and letting the model triage its own footage is a cheaper fix than a bigger context window.

For anyone already running video through the Gemini API, migration is one config field, so measure it on your own material rather than take the 88% at face value — that number is a ceiling from unnamed benchmarks. And if you have avoided video pipelines because hour-long files were too expensive to touch, $0.36 an hour was the number that stopped you, and it is no longer the number.

Sources

• Introducing agentic video understanding with Gemini — Google's announcement, September 1, 2026
• Video understanding — Gemini API docs: sampling rates, token costs per second, length limits, the processing parameter
• Gemini API pricing — introductory Flash rates and the January 1, 2027 change
• LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding — the benchmark named in Google's chart
• Gemini — our model page
• Google Gemini — the app and developer platform