Video Analyzer

Example

Minimal working example

action: llmvision.video_analyzer
data:
  provider: 01JAAFDSVEJEBMESBP62QP156T # Pick in UI
  message: Describe what happens in this video frame by frame.
  max_frames: 5
  max_tokens: 1000
  video_file: |-
    /config/www/tmp/front_door.mp4
  include_filename: false

Action Reference

You can call video_analyzer to analyze either local video files using the video_file or a Frigate event using event_id. You must pass at least one of the above parameters:

Action Reference

action: llmvision.video_analyzer
data:
  provider: 01J99F4T99PA1XGQ4CTQS3CP8H  # Select in UI dropdown
  message: What is happening in the video?
  max_tokens: 1000
  model: gpt-5-mini
  video_file: |-
    /config/www/tmp/front_door.mp4
    /config/www/tmp/garage.mp4
  event_id: 1712108310.968815-r28cdt
  max_frames: 5 # Analyze the 5 most relevant frames
  target_width: 1280
  include_filename: true

The provider id will not be the same for you. Switch to UI mode and select one of your configurations. If you don't see any, you need to set up at least one provider!

For all available models see: Choosing the right model

Parameter Reference

Parameter	Required	Description	Default
provider	Yes	The AI provider configuration
model	No	Model used for processing the image(s).
message	Yes	The prompt to send along with the image(s).
store_in_timeline	No	Add event to Timeline	false
use_memory	No	Use information stored in memory to provide additional context. Memory must be set up.	false
video_file	No*	The path to the video file(s). Each path must be on a new line.
event_id	No*	Event ID from Frigate. Each id must be on a new line.
max_frames	No	How many frames to analyze. Will pick the most relevant frames (most motion)	3
include_filename	Yes	Whether to include the filename in the request.	false
target_width	No	Width to downscale the image to before encoding.	1280
max_tokens	Yes	The maximum number of response tokens to generate.	1000
generate_title	No	Generate and returns a title in the response. (Used for notifications and remembered events)	false
expose_images	No	Save key frame to /config/media/llmvision/snapshots. File path is included in response as key_frame. If used together with remember images will be deleted after retention_time set in Timeline. Otherwise this folder will use a lot of disk space!	false
reponse_format	No	Format of the response - text for natural language or json for structured data
structure	No	JSON schema defining the expected response structure (only used when response_format is json). See <here> for more information.	See here
title_field	No	Name of the field in your JSON schema that contains the event title (used for timeline). Leave empty to use fallback title "Motion detected".	title
description_field	No	Name of the field in your JSON schema that contains the event description (used for timeline).	description